-- Logs begin at Sat 2022-08-27 01:35:59 UTC, end at Sat 2022-08-27 02:51:48 UTC. -- Aug 27 02:13:22.472792 dv-cinder-846303-3-iscsi systemd[1]: Starting Devstack devstack@n-api.service... Aug 27 02:13:22.487573 dv-cinder-846303-3-iscsi devstack@n-api.service[105768]: [uWSGI] getting INI configuration from /etc/nova/nova-api-uwsgi.ini Aug 27 02:13:22.487573 dv-cinder-846303-3-iscsi devstack@n-api.service[105768]: *** Starting uWSGI 2.0.18-debian (64bit) on [Sat Aug 27 02:13:22 2022] *** Aug 27 02:13:22.487573 dv-cinder-846303-3-iscsi devstack@n-api.service[105768]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Aug 27 02:13:22.487573 dv-cinder-846303-3-iscsi devstack@n-api.service[105768]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Aug 27 02:13:22.487573 dv-cinder-846303-3-iscsi devstack@n-api.service[105768]: nodename: dv-cinder-846303-3-iscsi Aug 27 02:13:22.487573 dv-cinder-846303-3-iscsi devstack@n-api.service[105768]: machine: x86_64 Aug 27 02:13:22.487573 dv-cinder-846303-3-iscsi devstack@n-api.service[105768]: clock source: unix Aug 27 02:13:22.487573 dv-cinder-846303-3-iscsi devstack@n-api.service[105768]: pcre jit disabled Aug 27 02:13:22.487573 dv-cinder-846303-3-iscsi devstack@n-api.service[105768]: detected number of CPU cores: 4 Aug 27 02:13:22.487573 dv-cinder-846303-3-iscsi devstack@n-api.service[105768]: current working directory: / Aug 27 02:13:22.487573 dv-cinder-846303-3-iscsi devstack@n-api.service[105768]: detected binary path: /usr/bin/uwsgi-core Aug 27 02:13:22.487573 dv-cinder-846303-3-iscsi devstack@n-api.service[105768]: your processes number limit is 47680 Aug 27 02:13:22.487573 dv-cinder-846303-3-iscsi devstack@n-api.service[105768]: your memory page size is 4096 bytes Aug 27 02:13:22.487573 dv-cinder-846303-3-iscsi devstack@n-api.service[105768]: detected max file descriptor number: 2048 Aug 27 02:13:22.487573 dv-cinder-846303-3-iscsi devstack@n-api.service[105768]: lock engine: pthread robust mutexes Aug 27 02:13:22.487573 dv-cinder-846303-3-iscsi devstack@n-api.service[105768]: thunder lock: enabled Aug 27 02:13:22.487573 dv-cinder-846303-3-iscsi devstack@n-api.service[105768]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/nova-api-wsgi.socket fd 4 Aug 27 02:13:22.487573 dv-cinder-846303-3-iscsi devstack@n-api.service[105768]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Aug 27 02:13:22.524597 dv-cinder-846303-3-iscsi devstack@n-api.service[105768]: Python main interpreter initialized at 0x56112db8a9c0 Aug 27 02:13:22.524597 dv-cinder-846303-3-iscsi devstack@n-api.service[105768]: python threads support enabled Aug 27 02:13:22.524597 dv-cinder-846303-3-iscsi devstack@n-api.service[105768]: your server socket listen backlog is limited to 100 connections Aug 27 02:13:22.524597 dv-cinder-846303-3-iscsi devstack@n-api.service[105768]: your mercy for graceful operations on workers is 90 seconds Aug 27 02:13:22.525041 dv-cinder-846303-3-iscsi devstack@n-api.service[105768]: mapped 403077 bytes (393 KB) for 2 cores Aug 27 02:13:22.525041 dv-cinder-846303-3-iscsi devstack@n-api.service[105768]: *** Operational MODE: preforking *** Aug 27 02:13:22.525321 dv-cinder-846303-3-iscsi devstack@n-api.service[105768]: *** uWSGI is running in multiple interpreter mode *** Aug 27 02:13:22.525321 dv-cinder-846303-3-iscsi devstack@n-api.service[105768]: spawned uWSGI master process (pid: 105768) Aug 27 02:13:22.525610 dv-cinder-846303-3-iscsi systemd[1]: Started Devstack devstack@n-api.service. Aug 27 02:13:22.529167 dv-cinder-846303-3-iscsi devstack@n-api.service[105768]: spawned uWSGI worker 1 (pid: 105769, cores: 1) Aug 27 02:13:22.529167 dv-cinder-846303-3-iscsi devstack@n-api.service[105768]: spawned uWSGI worker 2 (pid: 105770, cores: 1) Aug 27 02:13:22.529167 dv-cinder-846303-3-iscsi devstack@n-api.service[105768]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Aug 27 02:13:23.277356 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: Modules with known eventlet monkey patching issues were imported prior to eventlet monkey patching: urllib3. This warning can usually be ignored if the caller is only importing and not executing nova code. Aug 27 02:13:23.395886 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: Modules with known eventlet monkey patching issues were imported prior to eventlet monkey patching: urllib3. This warning can usually be ignored if the caller is only importing and not executing nova code. Aug 27 02:13:24.432757 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 27 02:13:24.432757 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: from cryptography.utils import int_from_bytes Aug 27 02:13:24.433719 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 27 02:13:24.433719 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: from cryptography.utils import int_from_bytes Aug 27 02:13:24.665281 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 27 02:13:24.665281 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: from cryptography.utils import int_from_bytes Aug 27 02:13:24.666007 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 27 02:13:24.666007 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: from cryptography.utils import int_from_bytes Aug 27 02:13:27.224570 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] ******************************************************************************** {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Aug 27 02:13:27.225216 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] Configuration options gathered from: {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Aug 27 02:13:27.225603 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] command line args: [] {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Aug 27 02:13:27.226019 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] config files: ['/etc/nova/api-paste.ini', '/etc/nova/nova.conf'] {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Aug 27 02:13:27.226315 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] ================================================================================ {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Aug 27 02:13:27.226895 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] allow_resize_to_same_host = True {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:13:27.227402 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] arq_binding_timeout = 300 {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:13:27.227828 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] block_device_allocate_retries = 60 {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:13:27.228194 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] block_device_allocate_retries_interval = 3 {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:13:27.228573 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] cert = self.pem {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:13:27.228930 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] compute_driver = None {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:13:27.229322 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] compute_monitors = [] {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:13:27.229690 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] config_dir = [] {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:13:27.230062 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] config_drive_format = iso9660 {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:13:27.230366 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] config_file = ['/etc/nova/api-paste.ini', '/etc/nova/nova.conf'] {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:13:27.230727 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] config_source = [] {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:13:27.231107 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] console_host = dv-cinder-846303-3-iscsi {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:13:27.231472 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] control_exchange = nova {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:13:27.231819 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] cpu_allocation_ratio = None {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:13:27.232169 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] daemon = False {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:13:27.232529 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] debug = True {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:13:27.232878 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] default_access_ip_network_name = None {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:13:27.233294 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] default_availability_zone = nova {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:13:27.233647 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] default_ephemeral_format = None {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:13:27.234195 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'glanceclient=WARN', 'oslo.privsep.daemon=INFO'] {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:13:27.234823 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] default_schedule_zone = None {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:13:27.235319 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] disk_allocation_ratio = None {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:13:27.235767 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] enable_new_services = True {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:13:27.236202 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] enabled_apis = ['osapi_compute'] {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:13:27.236614 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] enabled_ssl_apis = [] {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:13:27.237031 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] flat_injected = False {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:13:27.237487 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] force_config_drive = True {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:13:27.237959 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] force_raw_images = True {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:13:27.238370 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] graceful_shutdown_timeout = 5 {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:13:27.238779 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] heal_instance_info_cache_interval = 60 {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:13:27.239266 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] host = dv-cinder-846303-3-iscsi {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:13:27.239671 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] initial_cpu_allocation_ratio = 16.0 {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:13:27.240068 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] initial_disk_allocation_ratio = 1.0 {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:13:27.240478 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] initial_ram_allocation_ratio = 1.5 {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:13:27.240995 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] injected_network_template = /opt/stack/nova/nova/virt/interfaces.template {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:13:27.241454 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] instance_build_timeout = 0 {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:13:27.241858 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] instance_delete_interval = 300 {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:13:27.242295 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] instance_format = [instance: %(uuid)s] {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:13:27.242722 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] instance_name_template = instance-%08x {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:13:27.243124 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] instance_usage_audit = False {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:13:27.243566 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] instance_usage_audit_period = month {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:13:27.244195 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:13:27.244615 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] instances_path = /opt/stack/data/nova/instances {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:13:27.245051 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] internal_service_availability_zone = internal {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:13:27.245536 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] key = None {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:13:27.245935 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] live_migration_retry_count = 30 {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:13:27.246334 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] log_config_append = None {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:13:27.246780 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:13:27.247182 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] log_dir = None {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:13:27.247580 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] log_file = None {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:13:27.247962 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] log_options = True {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:13:27.248306 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] log_rotate_interval = 1 {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:13:27.248632 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] log_rotate_interval_type = days {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:13:27.248959 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] log_rotation_type = none {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:13:27.249316 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:13:27.249564 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:13:27.249813 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:13:27.250140 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:13:27.250464 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:13:27.250770 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] long_rpc_timeout = 1800 {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:13:27.251075 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_builds = 10 {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:13:27.251378 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_live_migrations = 1 {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:13:27.251679 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_snapshots = 5 {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:13:27.251983 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] max_local_block_devices = 3 {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:13:27.252285 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] max_logfile_count = 30 {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:13:27.252587 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] max_logfile_size_mb = 200 {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:13:27.252888 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] maximum_instance_delete_attempts = 5 {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:13:27.253362 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] ******************************************************************************** {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Aug 27 02:13:27.253817 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] metadata_listen = 0.0.0.0 {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:13:27.253968 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] Configuration options gathered from: {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Aug 27 02:13:27.254109 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] metadata_listen_port = 8775 {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:13:27.254109 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] metadata_workers = 2 {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:13:27.254402 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] command line args: [] {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Aug 27 02:13:27.254546 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] migrate_max_retries = -1 {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:13:27.254820 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] config files: ['/etc/nova/api-paste.ini', '/etc/nova/nova.conf'] {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Aug 27 02:13:27.254820 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] ================================================================================ {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Aug 27 02:13:27.255068 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] mkisofs_cmd = genisoimage {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:13:27.255153 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] allow_resize_to_same_host = True {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:13:27.255367 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] my_block_storage_ip = 10.222.0.52 {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:13:27.255470 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] arq_binding_timeout = 300 {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:13:27.255711 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] block_device_allocate_retries = 60 {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:13:27.255814 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] my_ip = 10.222.0.52 {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:13:27.256025 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] block_device_allocate_retries_interval = 3 {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:13:27.256124 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] network_allocate_retries = 0 {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:13:27.256356 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] cert = self.pem {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:13:27.256591 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] non_inheritable_image_properties = ['cache_in_nova', 'bittorrent'] {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:13:27.256677 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] compute_driver = None {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:13:27.256979 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] compute_monitors = [] {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:13:27.257067 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_listen = 0.0.0.0 {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:13:27.257320 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] config_dir = [] {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:13:27.257466 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_listen_port = 8774 {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:13:27.257651 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] config_drive_format = iso9660 {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:13:27.257946 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_unique_server_name_scope = {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:13:27.258068 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] config_file = ['/etc/nova/api-paste.ini', '/etc/nova/nova.conf'] {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:13:27.258376 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_workers = 2 {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:13:27.258741 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] password_length = 12 {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:13:27.258884 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] config_source = [] {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:13:27.259161 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] periodic_enable = True {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:13:27.259246 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] console_host = dv-cinder-846303-3-iscsi {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:13:27.259541 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] control_exchange = nova {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:13:27.259628 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] periodic_fuzzy_delay = 60 {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:13:27.259853 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] cpu_allocation_ratio = None {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:13:27.260010 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] pointer_model = usbtablet {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:13:27.260167 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] daemon = False {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:13:27.260460 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] preallocate_images = none {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:13:27.260600 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] debug = True {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:13:27.260782 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] default_access_ip_network_name = None {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:13:27.260875 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] publish_errors = False {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:13:27.261106 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] default_availability_zone = nova {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:13:27.261256 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] pybasedir = /opt/stack/nova {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:13:27.261440 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] default_ephemeral_format = None {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:13:27.261762 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] ram_allocation_ratio = None {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:13:27.261949 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'glanceclient=WARN', 'oslo.privsep.daemon=INFO'] {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:13:27.262175 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_burst = 0 {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:13:27.262260 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] default_schedule_zone = None {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:13:27.262556 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] disk_allocation_ratio = None {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:13:27.262654 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_except_level = CRITICAL {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:13:27.262865 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] enable_new_services = True {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:13:27.263030 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_interval = 0 {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:13:27.263200 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] enabled_apis = ['osapi_compute'] {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:13:27.263459 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] reboot_timeout = 0 {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:13:27.263551 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] enabled_ssl_apis = [] {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:13:27.263812 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] flat_injected = False {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:13:27.263898 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] reclaim_instance_interval = 0 {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:13:27.264128 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] force_config_drive = True {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:13:27.264254 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] record = None {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:13:27.264437 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] force_raw_images = True {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:13:27.264735 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] report_interval = 10 {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:13:27.264823 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] graceful_shutdown_timeout = 5 {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:13:27.265050 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] heal_instance_info_cache_interval = 60 {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:13:27.265164 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] rescue_timeout = 0 {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:13:27.265471 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] host = dv-cinder-846303-3-iscsi {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:13:27.265560 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_cpus = 0 {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:13:27.265787 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] initial_cpu_allocation_ratio = 16.0 {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:13:27.266005 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_disk_mb = 0 {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:13:27.266106 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] initial_disk_allocation_ratio = 1.0 {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:13:27.266393 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] initial_ram_allocation_ratio = 1.5 {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:13:27.266488 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_memory_mb = 512 {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:13:27.266804 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] reserved_huge_pages = None {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:13:27.266946 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] injected_network_template = /opt/stack/nova/nova/virt/interfaces.template {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:13:27.267211 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] resize_confirm_window = 0 {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:13:27.267305 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] instance_build_timeout = 0 {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:13:27.267557 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] instance_delete_interval = 300 {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:13:27.267684 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] resize_fs_using_block_device = False {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:13:27.267883 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] instance_format = [instance: %(uuid)s] {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:13:27.268042 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] resume_guests_state_on_host_boot = False {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:13:27.268217 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] instance_name_template = instance-%08x {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:13:27.268533 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] instance_usage_audit = False {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:13:27.268699 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] rootwrap_config = /etc/nova/rootwrap.conf {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:13:27.268841 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] instance_usage_audit_period = month {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:13:27.269167 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_action = reap {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:13:27.269319 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:13:27.269539 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] instances_path = /opt/stack/data/nova/instances {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:13:27.269663 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_poll_interval = 1800 {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:13:27.269866 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] internal_service_availability_zone = internal {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:13:27.270031 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_timeout = 0 {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:13:27.270166 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] key = None {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:13:27.270445 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] scheduler_instance_sync_interval = 120 {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:13:27.270539 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] live_migration_retry_count = 30 {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:13:27.270767 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] log_config_append = None {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:13:27.270862 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] service_down_time = 60 {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:13:27.271089 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:13:27.271298 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] servicegroup_driver = db {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:13:27.271381 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] log_dir = None {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:13:27.271684 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] log_file = None {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:13:27.271806 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] shelved_offload_time = 0 {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:13:27.271931 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] log_options = True {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:13:27.272212 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] shelved_poll_interval = 3600 {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:13:27.272311 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] log_rotate_interval = 1 {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:13:27.272547 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] log_rotate_interval_type = days {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:13:27.272656 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] shutdown_timeout = 0 {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:13:27.272870 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] log_rotation_type = none {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:13:27.273071 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] source_is_ipv6 = False {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:13:27.273216 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:13:27.273527 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:13:27.273628 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] ssl_only = False {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:13:27.273944 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] state_path = /opt/stack/data/nova {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:13:27.274313 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] sync_power_state_interval = 600 {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:13:27.274404 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:13:27.274695 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:13:27.274790 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] sync_power_state_pool_size = 1000 {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:13:27.275025 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:13:27.275122 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] syslog_log_facility = LOG_USER {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:13:27.275339 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] long_rpc_timeout = 1800 {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:13:27.275533 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] tempdir = None {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:13:27.275651 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_builds = 10 {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:13:27.275962 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_live_migrations = 1 {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:13:27.276057 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] timeout_nbd = 10 {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:13:27.276273 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_snapshots = 5 {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:13:27.276454 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] transport_url = **** {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:13:27.276585 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] max_local_block_devices = 3 {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:13:27.276867 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] update_resources_interval = 0 {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:13:27.276957 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] max_logfile_count = 30 {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:13:27.277214 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] max_logfile_size_mb = 200 {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:13:27.277326 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] use_cow_images = True {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:13:27.277524 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] maximum_instance_delete_attempts = 5 {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:13:27.277747 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] use_eventlog = False {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:13:27.277832 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] metadata_listen = 0.0.0.0 {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:13:27.278149 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] use_journal = False {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:13:27.278308 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] metadata_listen_port = 8775 {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:13:27.278480 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] metadata_workers = 2 {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:13:27.278623 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] use_json = False {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:13:27.278756 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] migrate_max_retries = -1 {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:13:27.279006 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] use_rootwrap_daemon = False {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:13:27.279173 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] mkisofs_cmd = genisoimage {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:13:27.279459 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] my_block_storage_ip = 10.222.0.52 {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:13:27.279634 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] use_stderr = False {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:13:27.279784 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] my_ip = 10.222.0.52 {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:13:27.280010 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] network_allocate_retries = 0 {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:13:27.280199 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] use_syslog = False {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:13:27.280343 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] non_inheritable_image_properties = ['cache_in_nova', 'bittorrent'] {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:13:27.280511 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] vcpu_pin_set = None {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:13:27.280661 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_listen = 0.0.0.0 {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:13:27.280905 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] vif_plugging_is_fatal = True {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:13:27.281049 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_listen_port = 8774 {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:13:27.281305 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_unique_server_name_scope = {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:13:27.281454 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] vif_plugging_timeout = 300 {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:13:27.281630 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_workers = 2 {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:13:27.281769 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] virt_mkfs = [] {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:13:27.281939 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] password_length = 12 {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:13:27.282236 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] volume_usage_poll_interval = 0 {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:13:27.282391 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] periodic_enable = True {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:13:27.282535 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] periodic_fuzzy_delay = 60 {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:13:27.282679 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] watch_log_file = False {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:13:27.282832 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] pointer_model = usbtablet {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:13:27.283100 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] web = /usr/share/spice-html5 {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:13:27.283242 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] preallocate_images = none {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:13:27.283471 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] publish_errors = False {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:13:27.283604 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] oslo_concurrency.disable_process_locking = False {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.283760 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] pybasedir = /opt/stack/nova {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:13:27.284003 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] oslo_concurrency.lock_path = /opt/stack/data/nova {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.284141 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] ram_allocation_ratio = None {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:13:27.284284 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_burst = 0 {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:13:27.284505 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] api.auth_strategy = keystone {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.284654 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_except_level = CRITICAL {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:13:27.284905 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_interval = 0 {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:13:27.285041 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] api.compute_link_prefix = None {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.285286 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] reboot_timeout = 0 {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:13:27.285426 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] api.config_drive_skip_versions = 1.0 2007-01-19 2007-03-01 2007-08-29 2007-10-10 2007-12-15 2008-02-01 2008-09-01 {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.285684 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] reclaim_instance_interval = 0 {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:13:27.285940 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] api.dhcp_domain = novalocal {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.286126 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] record = None {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:13:27.286270 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] report_interval = 10 {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:13:27.286400 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] api.enable_instance_password = True {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.286572 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] rescue_timeout = 0 {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:13:27.286808 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] api.glance_link_prefix = None {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.286973 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_cpus = 0 {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:13:27.287181 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_disk_mb = 0 {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:13:27.287320 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_cells_batch_fixed_size = 100 {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.287485 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_memory_mb = 512 {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:13:27.287695 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_cells_batch_strategy = distributed {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.287841 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] reserved_huge_pages = None {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:13:27.288080 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] resize_confirm_window = 0 {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:13:27.288172 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_per_project_cells = False {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.288385 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] resize_fs_using_block_device = False {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:13:27.288534 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] api.list_records_by_skipping_down_cells = True {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.288695 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] resume_guests_state_on_host_boot = False {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:13:27.288956 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] api.local_metadata_per_cell = False {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.289292 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] rootwrap_config = /etc/nova/rootwrap.conf {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:13:27.289678 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] api.max_limit = 1000 {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.290533 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] api.metadata_cache_expiration = 15 {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.291123 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_action = reap {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:13:27.291826 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] api.neutron_default_tenant_id = default {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.291826 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] api.use_forwarded_for = False {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.292371 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_poll_interval = 1800 {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:13:27.292520 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] api.use_neutron_default_nets = False {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.292667 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_timeout = 0 {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:13:27.292813 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_connect_timeout = 5 {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.292970 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] scheduler_instance_sync_interval = 120 {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:13:27.292970 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] service_down_time = 60 {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:13:27.293371 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_failure_fatal = False {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.293464 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] servicegroup_driver = db {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:13:27.293531 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_read_timeout = 5 {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.293595 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] shelved_offload_time = 0 {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:13:27.293770 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_ssl_certfile = {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.294132 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] shelved_poll_interval = 3600 {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:13:27.294277 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_targets = [] {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.294670 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] shutdown_timeout = 0 {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:13:27.294753 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_jsonfile_path = None {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.295088 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] source_is_ipv6 = False {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:13:27.295171 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_providers = ['StaticJSON'] {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.295550 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] ssl_only = False {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:13:27.295642 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] cache.backend = dogpile.cache.memcached {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.295988 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] cache.backend_argument = **** {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.296110 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] state_path = /opt/stack/data/nova {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:13:27.296473 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] cache.config_prefix = cache.oslo {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.296654 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] sync_power_state_interval = 600 {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:13:27.296921 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] cache.dead_timeout = 60.0 {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.297225 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] sync_power_state_pool_size = 1000 {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:13:27.297375 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] cache.debug_cache_backend = False {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.297747 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] syslog_log_facility = LOG_USER {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:13:27.297900 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] cache.enable_retry_client = False {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.298070 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] tempdir = None {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:13:27.298223 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] cache.enable_socket_keepalive = False {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.298492 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] timeout_nbd = 10 {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:13:27.298775 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] cache.enabled = True {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.298938 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] transport_url = **** {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:13:27.299233 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] cache.expiration_time = 600 {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.299333 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] update_resources_interval = 0 {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:13:27.299603 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] use_cow_images = True {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:13:27.299763 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] cache.hashclient_retry_attempts = 2 {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.300011 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] use_eventlog = False {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:13:27.300201 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] cache.hashclient_retry_delay = 1.0 {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.300402 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] use_journal = False {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:13:27.300637 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_dead_retry = 300 {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.300756 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] use_json = False {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:13:27.301028 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] use_rootwrap_daemon = False {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:13:27.301262 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_password = {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.301440 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] use_stderr = False {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:13:27.301588 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.301739 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] use_syslog = False {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:13:27.301981 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.302122 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] vcpu_pin_set = None {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:13:27.302312 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] vif_plugging_is_fatal = True {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:13:27.302478 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_maxsize = 10 {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.302647 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] vif_plugging_timeout = 300 {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:13:27.302915 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_unused_timeout = 60 {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.303067 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] virt_mkfs = [] {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:13:27.303283 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] volume_usage_poll_interval = 0 {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:13:27.303422 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_sasl_enabled = False {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.303602 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] watch_log_file = False {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:13:27.303831 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_servers = ['localhost:11211'] {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.304026 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] web = /usr/share/spice-html5 {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:13:27.304285 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_socket_timeout = 1.0 {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.304426 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] oslo_concurrency.disable_process_locking = False {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.304882 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] oslo_concurrency.lock_path = /opt/stack/data/nova {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.305030 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_username = {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.305270 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] api.auth_strategy = keystone {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.305399 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] cache.proxies = [] {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.305759 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] api.compute_link_prefix = None {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.305901 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] cache.retry_attempts = 2 {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.306118 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] api.config_drive_skip_versions = 1.0 2007-01-19 2007-03-01 2007-08-29 2007-10-10 2007-12-15 2008-02-01 2008-09-01 {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.306283 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] cache.retry_delay = 0.0 {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.306470 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] api.dhcp_domain = novalocal {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.306694 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_count = 1 {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.306851 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] api.enable_instance_password = True {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.307119 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] api.glance_link_prefix = None {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.307259 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_idle = 1 {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.307445 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_cells_batch_fixed_size = 100 {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.307600 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_interval = 1 {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.307850 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_cells_batch_strategy = distributed {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.307991 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_allowed_ciphers = None {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.308179 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_per_project_cells = False {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.308396 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_cafile = None {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.308538 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] api.list_records_by_skipping_down_cells = True {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.308812 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] api.local_metadata_per_cell = False {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.308961 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_certfile = None {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.309157 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] api.max_limit = 1000 {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.309299 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_enabled = False {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.309499 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] api.metadata_cache_expiration = 15 {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.309733 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_keyfile = None {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.309894 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] api.neutron_default_tenant_id = default {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.310167 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] api.use_forwarded_for = False {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.310327 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] cinder.auth_section = None {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.310492 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] api.use_neutron_default_nets = False {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.310746 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] cinder.auth_type = password {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.310898 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_connect_timeout = 5 {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.311179 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] cinder.cafile = None {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.311380 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_failure_fatal = False {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.311518 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_read_timeout = 5 {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.311710 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] cinder.catalog_info = volumev3::publicURL {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.311859 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_ssl_certfile = {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.312146 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] cinder.certfile = None {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.312288 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_targets = [] {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.312488 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_jsonfile_path = None {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.312625 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] cinder.collect_timing = False {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.312845 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_providers = ['StaticJSON'] {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.313019 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] cinder.cross_az_attach = True {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.313215 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] cache.backend = dogpile.cache.memcached {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.313816 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] cinder.debug = False {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.313959 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] cache.backend_argument = **** {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.314157 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] cache.config_prefix = cache.oslo {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.314313 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] cinder.endpoint_template = None {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.314522 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] cache.dead_timeout = 60.0 {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.314742 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] cinder.http_retries = 3 {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.314884 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] cache.debug_cache_backend = False {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.315101 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] cache.enable_retry_client = False {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.315255 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] cinder.insecure = False {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.315420 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] cache.enable_socket_keepalive = False {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.315684 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] cinder.keyfile = None {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.315824 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] cache.enabled = True {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.316189 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] cache.expiration_time = 600 {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.316335 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] cinder.os_region_name = RegionOne {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.316516 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] cache.hashclient_retry_attempts = 2 {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.316656 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] cinder.split_loggers = False {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.316969 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] cache.hashclient_retry_delay = 1.0 {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.317111 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] cinder.timeout = None {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.317522 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] compute.consecutive_build_service_disable_threshold = 10 {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.317609 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_dead_retry = 300 {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.317925 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] compute.cpu_dedicated_set = None {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.318154 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_password = {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.318349 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] compute.cpu_shared_set = None {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.318732 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.318817 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] compute.image_type_exclude_list = [] {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.319142 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] compute.live_migration_wait_for_vif_plug = True {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.319310 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.319574 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] compute.max_concurrent_disk_ops = 0 {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.319906 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_maxsize = 10 {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.320066 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] compute.max_disk_devices_to_attach = -1 {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.320558 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] compute.packing_host_numa_cells_allocation_strategy = False {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.320715 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_unused_timeout = 60 {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.321035 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] compute.provider_config_location = /etc/nova/provider_config/ {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.321172 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_sasl_enabled = False {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.321495 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] compute.resource_provider_association_refresh = 300 {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.321772 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_servers = ['localhost:11211'] {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.321881 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] compute.shutdown_retry_interval = 10 {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.322278 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] conductor.workers = 2 {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.322370 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_socket_timeout = 1.0 {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.322706 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] console.allowed_origins = [] {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.322950 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_username = {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.323118 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] console.ssl_ciphers = None {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.323521 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] console.ssl_minimum_version = default {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.323612 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] cache.proxies = [] {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.323948 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] consoleauth.token_ttl = 600 {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.324106 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] cache.retry_attempts = 2 {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.324386 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.cafile = None {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.324704 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] cache.retry_delay = 0.0 {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.324850 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.certfile = None {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.325321 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_count = 1 {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.325430 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.collect_timing = False {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.325777 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.connect_retries = None {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.325908 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_idle = 1 {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.326193 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.connect_retry_delay = None {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.326486 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_interval = 1 {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.326571 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.endpoint_override = None {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.327020 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.insecure = False {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.327105 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_allowed_ciphers = None {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.327425 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.keyfile = None {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.327612 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_cafile = None {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.327849 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.max_version = None {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.328180 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_certfile = None {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.328272 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.min_version = None {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.328605 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.region_name = None {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.328760 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_enabled = False {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.329027 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.service_name = None {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.329355 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_keyfile = None {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.329488 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.service_type = accelerator {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.329881 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.split_loggers = False {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.329967 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] cinder.auth_section = None {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.330289 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.status_code_retries = None {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.330542 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] cinder.auth_type = password {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.330720 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.status_code_retry_delay = None {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.331089 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.timeout = None {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.331192 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] cinder.cafile = None {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.331533 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.valid_interfaces = ['internal', 'public'] {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.331724 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] cinder.catalog_info = volumev3::publicURL {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.332084 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.version = None {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.332287 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] cinder.certfile = None {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.332539 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] database.backend = sqlalchemy {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.332868 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] cinder.collect_timing = False {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.332961 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] database.connection = **** {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.333379 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_debug = 0 {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.333491 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] cinder.cross_az_attach = True {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.333821 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_parameters = {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.334074 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] cinder.debug = False {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.334245 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_recycle_time = 3600 {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.334630 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_trace = False {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.334713 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] cinder.endpoint_template = None {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.335094 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] database.db_inc_retry_interval = True {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.335187 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] cinder.http_retries = 3 {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.335559 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] database.db_max_retries = 20 {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.335767 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] cinder.insecure = False {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.336083 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] database.db_max_retry_interval = 10 {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.336399 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] cinder.keyfile = None {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.336565 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] database.db_retry_interval = 1 {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.336873 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] database.max_overflow = 50 {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.337014 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] cinder.os_region_name = RegionOne {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.337352 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] database.max_pool_size = 5 {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.337580 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] cinder.split_loggers = False {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.337792 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] database.max_retries = 10 {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.338134 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] cinder.timeout = None {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.338237 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] database.mysql_enable_ndb = False {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.338578 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] database.mysql_sql_mode = TRADITIONAL {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.338729 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] compute.consecutive_build_service_disable_threshold = 10 {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.339002 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] database.pool_timeout = None {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.339297 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] compute.cpu_dedicated_set = None {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.339419 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] database.retry_interval = 10 {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.339803 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] database.slave_connection = **** {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.339887 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] compute.cpu_shared_set = None {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.340226 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] database.sqlite_synchronous = True {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.340438 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] compute.image_type_exclude_list = [] {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.340753 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] api_database.backend = sqlalchemy {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.341010 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] compute.live_migration_wait_for_vif_plug = True {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.341230 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection = **** {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.341618 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] compute.max_concurrent_disk_ops = 0 {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.341715 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_debug = 0 {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.342034 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_parameters = {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.342192 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] compute.max_disk_devices_to_attach = -1 {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.342467 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_recycle_time = 3600 {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.342769 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] compute.packing_host_numa_cells_allocation_strategy = False {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.342855 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_trace = False {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.343305 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_inc_retry_interval = True {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.343457 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] compute.provider_config_location = /etc/nova/provider_config/ {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.343877 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_max_retries = 20 {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.344038 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] compute.resource_provider_association_refresh = 300 {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.344385 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_max_retry_interval = 10 {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.344480 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] compute.shutdown_retry_interval = 10 {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.344836 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_retry_interval = 1 {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.345077 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] conductor.workers = 2 {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.345328 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_overflow = 50 {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.345720 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] console.allowed_origins = [] {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.345807 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_pool_size = 5 {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.346168 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_retries = 10 {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.346285 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] console.ssl_ciphers = None {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.346595 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] api_database.mysql_enable_ndb = False {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.346883 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] console.ssl_minimum_version = default {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.347016 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] api_database.mysql_sql_mode = TRADITIONAL {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.347421 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] api_database.pool_timeout = None {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.347509 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] consoleauth.token_ttl = 600 {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.347837 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] api_database.retry_interval = 10 {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.348044 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.cafile = None {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.348257 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] api_database.slave_connection = **** {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.348604 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.certfile = None {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.348690 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] api_database.sqlite_synchronous = True {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.349061 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] devices.enabled_mdev_types = [] {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.349195 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.collect_timing = False {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.349658 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.cipher = aes-xts-plain64 {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.349779 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.connect_retries = None {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.350088 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.enabled = False {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.350315 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.connect_retry_delay = None {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.350529 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.key_size = 512 {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.350865 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.endpoint_override = None {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.350951 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] glance.api_servers = None {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.351333 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] glance.cafile = None {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.351446 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.insecure = False {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.351774 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] glance.certfile = None {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.352152 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] glance.collect_timing = False {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.352530 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] glance.connect_retries = None {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.352897 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] glance.connect_retry_delay = None {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.353312 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] glance.debug = False {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.353484 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.keyfile = None {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.353752 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] glance.default_trusted_certificate_ids = [] {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.354067 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.max_version = None {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.354152 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] glance.enable_certificate_validation = False {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.354518 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] glance.enable_rbd_download = False {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.354644 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.min_version = None {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.354939 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] glance.endpoint_override = None {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.355221 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.region_name = None {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.355422 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] glance.insecure = False {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.355797 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.service_name = None {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.355895 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] glance.keyfile = None {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.356256 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] glance.max_version = None {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.356404 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.service_type = accelerator {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.356675 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] glance.min_version = None {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.356993 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.split_loggers = False {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.357167 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] glance.num_retries = 3 {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.357524 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_ceph_conf = {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.357612 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.status_code_retries = None {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.357934 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_connect_timeout = 5 {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.358145 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.status_code_retry_delay = None {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.358378 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_pool = {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.358710 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.timeout = None {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.358803 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_user = {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.359168 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] glance.region_name = None {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.359349 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.valid_interfaces = ['internal', 'public'] {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.359595 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] glance.service_name = None {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.359917 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.version = None {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.360008 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] glance.service_type = image {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.360390 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] glance.split_loggers = False {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.360541 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] database.backend = sqlalchemy {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.360867 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] glance.status_code_retries = None {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.361186 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] database.connection = **** {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.361336 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] glance.status_code_retry_delay = None {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.361693 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] glance.timeout = None {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.361843 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_debug = 0 {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.362164 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] glance.valid_interfaces = ['internal', 'public'] {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.362358 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_parameters = {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.362592 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] glance.verify_glance_signatures = False {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.362938 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_recycle_time = 3600 {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.363033 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] glance.version = None {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.363370 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] guestfs.debug = False {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.363520 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_trace = False {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.363949 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.config_drive_cdrom = False {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.364101 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] database.db_inc_retry_interval = True {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.364380 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.config_drive_inject_password = False {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.364714 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] database.db_max_retries = 20 {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.364867 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.dynamic_memory_ratio = 1.0 {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.365319 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] database.db_max_retry_interval = 10 {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.365433 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.enable_instance_metrics_collection = False {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.365783 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.enable_remotefx = False {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.365921 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] database.db_retry_interval = 1 {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.366249 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.instances_path_share = {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.366462 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] database.max_overflow = 50 {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.366684 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.iscsi_initiator_list = [] {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.367046 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] database.max_pool_size = 5 {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.367341 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.limit_cpu_features = False {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.367691 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] database.max_retries = 10 {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.367855 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.mounted_disk_query_retry_count = 10 {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.368269 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] database.mysql_enable_ndb = False {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.368414 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.mounted_disk_query_retry_interval = 5 {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.368708 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.power_state_check_timeframe = 60 {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.368877 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] database.mysql_sql_mode = TRADITIONAL {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.369183 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.power_state_event_polling_interval = 2 {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.369484 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] database.pool_timeout = None {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.369684 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.qemu_img_cmd = qemu-img.exe {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.370050 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.use_multipath_io = False {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.370177 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] database.retry_interval = 10 {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.370480 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.volume_attach_retry_count = 10 {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.370623 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] database.slave_connection = **** {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.370974 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.volume_attach_retry_interval = 5 {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.371199 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] database.sqlite_synchronous = True {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.371398 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.vswitch_name = None {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.371781 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.wait_soft_reboot_seconds = 60 {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.371924 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] api_database.backend = sqlalchemy {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.372223 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] mks.enabled = False {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.372426 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection = **** {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.372991 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] mks.mksproxy_base_url = http://127.0.0.1:6090/ {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.373184 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_debug = 0 {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.373513 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.manager_interval = 2400 {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.373634 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_parameters = {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.373944 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.precache_concurrency = 1 {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.374189 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_recycle_time = 3600 {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.374387 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_base_images = True {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.374772 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_trace = False {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.374858 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_original_minimum_age_seconds = 86400 {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.375212 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_resized_minimum_age_seconds = 3600 {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.375348 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_inc_retry_interval = True {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.375768 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.subdirectory_name = _base {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.375929 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_max_retries = 20 {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.376288 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] ironic.api_max_retries = 60 {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.376503 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_max_retry_interval = 10 {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.376725 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] ironic.api_retry_interval = 2 {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.377087 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_retry_interval = 1 {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.377247 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] ironic.auth_section = None {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.377663 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] ironic.auth_type = None {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.377753 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_overflow = 50 {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.378073 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] ironic.cafile = None {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.378352 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_pool_size = 5 {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.378670 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] ironic.certfile = None {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.378939 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] ironic.collect_timing = False {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.379096 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_retries = 10 {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.379327 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] ironic.connect_retries = None {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.379483 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] api_database.mysql_enable_ndb = False {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.380050 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] ironic.connect_retry_delay = None {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.380162 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] api_database.mysql_sql_mode = TRADITIONAL {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.380690 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] ironic.endpoint_override = None {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.380795 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] api_database.pool_timeout = None {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.380863 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] ironic.insecure = False {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.381108 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] ironic.keyfile = None {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.381280 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] api_database.retry_interval = 10 {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.381583 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] ironic.max_version = None {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.381683 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] api_database.slave_connection = **** {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.381994 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] ironic.min_version = None {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.382257 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] api_database.sqlite_synchronous = True {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.382392 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] ironic.partition_key = None {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.382927 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] ironic.peer_list = [] {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.383218 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] ironic.region_name = None {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.383326 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] devices.enabled_mdev_types = [] {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.383669 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] ironic.serial_console_state_timeout = 10 {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.383931 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.cipher = aes-xts-plain64 {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.384080 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] ironic.service_name = None {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.384502 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] ironic.service_type = baremetal {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.384613 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.enabled = False {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.384921 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] ironic.split_loggers = False {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.385073 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.key_size = 512 {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.385366 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] ironic.status_code_retries = None {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.385699 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] glance.api_servers = None {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.385872 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] ironic.status_code_retry_delay = None {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.386290 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] glance.cafile = None {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.386477 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] ironic.timeout = None {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.386813 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] ironic.valid_interfaces = ['internal', 'public'] {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.386943 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] glance.certfile = None {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.387214 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] ironic.version = None {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.387441 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] glance.collect_timing = False {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.387697 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] key_manager.backend = nova.keymgr.conf_key_mgr.ConfKeyManager {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.388006 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] glance.connect_retries = None {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.388145 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] key_manager.fixed_key = **** {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.388577 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] glance.connect_retry_delay = None {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.388669 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] barbican.auth_endpoint = http://localhost/identity/v3 {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.388989 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_api_version = None {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.389176 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] glance.debug = False {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.389445 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_endpoint = None {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.389766 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] glance.default_trusted_certificate_ids = [] {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.389870 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_endpoint_type = public {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.390229 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_region_name = None {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.390319 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] glance.enable_certificate_validation = False {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.390639 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] barbican.cafile = None {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.390906 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] glance.enable_rbd_download = False {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.391040 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] barbican.certfile = None {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.391441 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] barbican.collect_timing = False {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.391546 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] glance.endpoint_override = None {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.392051 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] barbican.insecure = False {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.392200 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] glance.insecure = False {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.392310 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] barbican.keyfile = None {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.392612 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] glance.keyfile = None {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.392720 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] barbican.number_of_retries = 60 {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.393184 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] barbican.retry_delay = 1 {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.393368 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] glance.max_version = None {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.393719 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] barbican.send_service_user_token = False {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.393875 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] glance.min_version = None {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.394257 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] barbican.split_loggers = False {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.394419 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] glance.num_retries = 3 {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.394781 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] barbican.timeout = None {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.394949 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_ceph_conf = {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.395322 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] barbican.verify_ssl = True {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.395475 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_connect_timeout = 5 {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.396070 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_pool = {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.396244 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] barbican.verify_ssl_path = None {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.396635 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.auth_section = None {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.396819 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_user = {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.397103 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.auth_type = None {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.397264 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] glance.region_name = None {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.397544 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.cafile = None {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.397812 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] glance.service_name = None {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.398752 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.certfile = None {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.398907 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] glance.service_type = image {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.399039 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.collect_timing = False {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.399154 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] glance.split_loggers = False {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.399448 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.insecure = False {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.399598 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] glance.status_code_retries = None {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.400009 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.keyfile = None {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.400092 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] glance.status_code_retry_delay = None {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.400565 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.split_loggers = False {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.400644 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] glance.timeout = None {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.401098 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.timeout = None {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.401392 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] glance.valid_interfaces = ['internal', 'public'] {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.401702 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] vault.approle_role_id = None {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.401865 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] glance.verify_glance_signatures = False {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.402231 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] vault.approle_secret_id = None {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.402419 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] glance.version = None {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.402784 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] vault.cafile = None {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.403012 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] guestfs.debug = False {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.403341 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] vault.certfile = None {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.403606 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.config_drive_cdrom = False {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.403883 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] vault.collect_timing = False {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.404180 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.config_drive_inject_password = False {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.404294 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] vault.insecure = False {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.404688 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] vault.keyfile = None {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.404790 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.dynamic_memory_ratio = 1.0 {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.405137 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] vault.kv_mountpoint = secret {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.405364 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.enable_instance_metrics_collection = False {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.405598 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] vault.kv_version = 2 {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.405940 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.enable_remotefx = False {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.406052 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] vault.namespace = None {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.406435 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] vault.root_token_id = None {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.406539 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.instances_path_share = {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.406900 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] vault.split_loggers = False {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.407109 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.iscsi_initiator_list = [] {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.407335 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] vault.ssl_ca_crt_file = None {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.407685 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.limit_cpu_features = False {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.407888 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] vault.timeout = None {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.408257 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.mounted_disk_query_retry_count = 10 {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.408370 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] vault.use_ssl = False {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.408690 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] vault.vault_url = http://127.0.0.1:8200 {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.408801 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.mounted_disk_query_retry_interval = 5 {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.409174 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] keystone.cafile = None {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.409408 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.power_state_check_timeframe = 60 {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.409557 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] keystone.certfile = None {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.409956 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] keystone.collect_timing = False {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.410058 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.power_state_event_polling_interval = 2 {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.410361 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] keystone.connect_retries = None {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.410555 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.qemu_img_cmd = qemu-img.exe {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.410788 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] keystone.connect_retry_delay = None {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.411129 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.use_multipath_io = False {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.411253 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] keystone.endpoint_override = None {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.411634 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] keystone.insecure = False {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.411729 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.volume_attach_retry_count = 10 {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.412049 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] keystone.keyfile = None {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.412329 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.volume_attach_retry_interval = 5 {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.412515 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] keystone.max_version = None {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.412810 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.vswitch_name = None {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.412963 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] keystone.min_version = None {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.413389 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] keystone.region_name = None {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.413562 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.wait_soft_reboot_seconds = 60 {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.413794 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] keystone.service_name = None {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.413982 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] mks.enabled = False {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.414367 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] keystone.service_type = identity {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.414928 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] keystone.split_loggers = False {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.415101 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] mks.mksproxy_base_url = http://127.0.0.1:6090/ {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.415361 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] keystone.status_code_retries = None {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.415732 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] keystone.status_code_retry_delay = None {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.415958 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.manager_interval = 2400 {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.416160 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] keystone.timeout = None {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.416540 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.precache_concurrency = 1 {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.416659 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] keystone.valid_interfaces = ['internal', 'public'] {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.417070 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] keystone.version = None {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.417212 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_base_images = True {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.417566 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.connection_uri = {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.417736 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_original_minimum_age_seconds = 86400 {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.417987 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_mode = None {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.418372 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_model_extra_flags = [] {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.418475 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_resized_minimum_age_seconds = 3600 {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.418799 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_models = [] {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.418982 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.subdirectory_name = _base {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.419222 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.device_detach_attempts = 8 {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.419584 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] ironic.api_max_retries = 60 {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.419681 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.device_detach_timeout = 20 {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.419997 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.disk_cachemodes = [] {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.420177 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] ironic.api_retry_interval = 2 {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.420474 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.disk_prefix = None {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.420723 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] ironic.auth_section = None {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.420904 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.enabled_perf_events = [] {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.421328 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] ironic.auth_type = None {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.421415 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.file_backed_memory = 0 {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.421742 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.gid_maps = [] {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.421896 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] ironic.cafile = None {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.422171 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.hw_disk_discard = None {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.422466 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] ironic.certfile = None {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.422656 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.hw_machine_type = None {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.423046 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] ironic.collect_timing = False {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.423139 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_ceph_conf = {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.423603 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_copy_poll_interval = 15 {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.423761 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] ironic.connect_retries = None {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.424029 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_copy_timeout = 600 {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.424180 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] ironic.connect_retry_delay = None {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.424472 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_store_name = {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.424712 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] ironic.endpoint_override = None {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.424910 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_pool = rbd {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.431606 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] ironic.insecure = False {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.431813 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_type = default {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.431813 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_volume_group = None {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.432064 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] ironic.keyfile = None {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.432197 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_key = False {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.432197 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_partition = -2 {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.432490 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] ironic.max_version = None {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.432645 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_password = False {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.432645 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.iscsi_iface = None {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.432890 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] ironic.min_version = None {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.433018 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.iser_use_multipath = False {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.433204 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] ironic.partition_key = None {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.433362 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_bandwidth = 0 {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.433362 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_completion_timeout = 800 {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.433612 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] ironic.peer_list = [] {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.433741 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime = 500 {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.433741 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime_delay = 75 {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.433986 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] ironic.region_name = None {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.434124 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime_steps = 10 {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.434124 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_inbound_addr = None {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.434357 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] ironic.serial_console_state_timeout = 10 {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.434483 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_permit_auto_converge = False {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.434483 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_permit_post_copy = False {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.434483 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_scheme = None {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.434888 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] ironic.service_name = None {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.435023 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_timeout_action = abort {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.435023 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_tunnelled = False {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.435023 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_uri = None {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.435364 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] ironic.service_type = baremetal {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.435507 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_with_native_tls = False {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.435507 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.max_queues = None {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.435751 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] ironic.split_loggers = False {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.435879 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.mem_stats_period_seconds = 10 {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.436002 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] ironic.status_code_retries = None {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.436127 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.nfs_mount_options = None {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.436267 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] ironic.status_code_retry_delay = None {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.436267 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] ironic.timeout = None {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.436524 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.nfs_mount_point_base = /opt/stack/data/nova/mnt {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.436524 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_aoe_discover_tries = 3 {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.436753 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] ironic.valid_interfaces = ['internal', 'public'] {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.436927 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_iser_scan_tries = 5 {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.437054 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] ironic.version = None {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.437054 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] key_manager.backend = nova.keymgr.conf_key_mgr.ConfKeyManager {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.437331 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_memory_encrypted_guests = None {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.437455 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] key_manager.fixed_key = **** {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.437578 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_nvme_discover_tries = 5 {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.437734 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] barbican.auth_endpoint = http://localhost/identity/v3 {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.437861 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_pcie_ports = 0 {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.437985 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_api_version = None {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.438108 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_volume_scan_tries = 5 {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.438234 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_endpoint = None {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.438667 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.pmem_namespaces = [] {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.438667 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.quobyte_client_cfg = None {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.438922 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_endpoint_type = public {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.438922 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_region_name = None {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.439150 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.quobyte_mount_point_base = /opt/stack/data/nova/mnt {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.439527 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] barbican.cafile = None {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.439678 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_connect_timeout = 5 {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.440051 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] barbican.certfile = None {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.440218 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_destroy_volume_retries = 12 {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.440550 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_destroy_volume_retry_interval = 5 {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.440713 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] barbican.collect_timing = False {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.441024 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_secret_uuid = None {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.441233 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] barbican.insecure = False {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.441527 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_user = None {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.441742 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] barbican.keyfile = None {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.442043 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.realtime_scheduler_priority = 1 {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.442325 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] barbican.number_of_retries = 60 {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.442572 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.remote_filesystem_transport = ssh {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.442899 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] barbican.retry_delay = 1 {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.443057 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_image_id = None {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.443478 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] barbican.send_service_user_token = False {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.443627 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_kernel_id = None {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.443956 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_ramdisk_id = None {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.444101 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] barbican.split_loggers = False {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.444461 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rng_dev_path = /dev/urandom {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.444609 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] barbican.timeout = None {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.444941 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rx_queue_size = None {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.445170 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] barbican.verify_ssl = True {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.445542 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.smbfs_mount_options = {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.445813 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] barbican.verify_ssl_path = None {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.446384 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.smbfs_mount_point_base = /opt/stack/data/nova/mnt {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.446528 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.auth_section = None {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.446894 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshot_compression = False {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.447049 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.auth_type = None {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.447347 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshot_image_format = None {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.447515 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.cafile = None {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.448048 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshots_directory = /opt/stack/data/nova/instances/snapshots {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.448184 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.certfile = None {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.448495 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.sparse_logical_volumes = False {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.448652 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.collect_timing = False {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.448918 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_enabled = False {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.449200 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.insecure = False {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.449402 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_group = tss {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.449763 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.keyfile = None {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.449948 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_user = tss {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.450333 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.split_loggers = False {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.450440 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.sysinfo_serial = unique {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.450746 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.tx_queue_size = None {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.450887 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.timeout = None {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.451176 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.uid_maps = [] {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.451464 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] vault.approle_role_id = None {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.451582 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.use_virtio_for_bridges = True {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.451991 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.virt_type = kvm {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.452095 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] vault.approle_secret_id = None {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.452416 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_clear = zero {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.452567 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] vault.cafile = None {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.452843 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_clear_size = 0 {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.453144 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] vault.certfile = None {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.453292 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_use_multipath = False {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.453693 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_cache_path = None {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.453858 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] vault.collect_timing = False {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.454140 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_log_path = /var/log/vstorage/%(cluster_name)s/nova.log.gz {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.454270 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] vault.insecure = False {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.454593 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_group = qemu {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.454838 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] vault.keyfile = None {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.455171 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_opts = [] {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.455444 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] vault.kv_mountpoint = secret {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.455583 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_perms = 0770 {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.456023 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] vault.kv_version = 2 {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.456165 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_point_base = /opt/stack/data/nova/mnt {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.456525 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_user = stack {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.456673 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] vault.namespace = None {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.456956 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.wait_soft_reboot_seconds = 120 {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.457106 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] vault.root_token_id = None {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.457453 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] neutron.auth_section = None {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.457718 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] vault.split_loggers = False {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.457864 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] neutron.auth_type = password {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.458255 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] neutron.cafile = None {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.458395 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] vault.ssl_ca_crt_file = None {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.458674 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] neutron.certfile = None {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.458819 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] vault.timeout = None {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.459097 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] neutron.collect_timing = False {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.459371 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] vault.use_ssl = False {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.459491 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] neutron.connect_retries = None {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.459965 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] vault.vault_url = http://127.0.0.1:8200 {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.460099 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] neutron.connect_retry_delay = None {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.460413 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] neutron.default_floating_pool = public {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.460574 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] keystone.cafile = None {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.460831 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] neutron.endpoint_override = None {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.461233 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] keystone.certfile = None {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.461375 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] neutron.extension_sync_interval = 600 {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.461745 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] neutron.http_retries = 3 {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.462005 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] keystone.collect_timing = False {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.462094 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] neutron.insecure = False {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.462362 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] keystone.connect_retries = None {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.462545 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] neutron.keyfile = None {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.462921 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] neutron.max_version = None {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.463005 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] keystone.connect_retry_delay = None {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.463344 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] neutron.metadata_proxy_shared_secret = **** {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.463481 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] keystone.endpoint_override = None {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.463768 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] neutron.min_version = None {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.464033 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] keystone.insecure = False {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.464261 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] neutron.ovs_bridge = br-int {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.464595 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] keystone.keyfile = None {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.464756 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] neutron.physnets = [] {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.465177 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] keystone.max_version = None {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.465264 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] neutron.region_name = RegionOne {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.465600 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_metadata_proxy = True {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.465739 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] keystone.min_version = None {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.466023 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_name = None {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.466308 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] keystone.region_name = None {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.466453 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_type = network {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.466835 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] neutron.split_loggers = False {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.466920 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] keystone.service_name = None {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.467245 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] neutron.status_code_retries = None {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.467457 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] keystone.service_type = identity {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.467665 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] neutron.status_code_retry_delay = None {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.468030 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] keystone.split_loggers = False {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.468123 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] neutron.timeout = None {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.468469 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] neutron.valid_interfaces = ['internal', 'public'] {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.468594 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] keystone.status_code_retries = None {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.468891 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] neutron.version = None {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.469188 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] keystone.status_code_retry_delay = None {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.469364 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] notifications.bdms_in_notifications = False {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.469751 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] keystone.timeout = None {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.469835 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] notifications.default_level = INFO {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.470269 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] notifications.notification_format = unversioned {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.470371 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] keystone.valid_interfaces = ['internal', 'public'] {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.470688 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] notifications.notify_on_state_change = None {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.470937 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] keystone.version = None {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.471140 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] notifications.versioned_notifications_topics = ['versioned_notifications'] {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.471534 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] pci.alias = [] {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.471627 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.connection_uri = {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.472634 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] pci.device_spec = [] {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.477591 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_mode = None {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.477739 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] placement.auth_section = None {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.477885 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_model_extra_flags = [] {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.478012 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] placement.auth_type = password {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.478012 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] placement.cafile = None {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.478269 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_models = [] {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.478392 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] placement.certfile = None {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.478392 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] placement.collect_timing = False {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.478610 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.device_detach_attempts = 8 {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.478733 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] placement.connect_retries = None {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.478733 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] placement.connect_retry_delay = None {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.478981 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.device_detach_timeout = 20 {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.479089 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] placement.endpoint_override = None {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.479210 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.disk_cachemodes = [] {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.479322 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] placement.insecure = False {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.479322 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] placement.keyfile = None {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.479577 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.disk_prefix = None {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.479764 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] placement.max_version = None {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.479764 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] placement.min_version = None {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.480021 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.enabled_perf_events = [] {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.480123 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] placement.region_name = RegionOne {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.480123 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] placement.service_name = None {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.480123 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] placement.service_type = placement {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.480123 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] placement.split_loggers = False {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.480123 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] placement.status_code_retries = None {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.480123 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] placement.status_code_retry_delay = None {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.480717 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.file_backed_memory = 0 {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.480839 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] placement.timeout = None {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.480965 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.gid_maps = [] {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.481090 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] placement.valid_interfaces = ['internal', 'public'] {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.481262 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.hw_disk_discard = None {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.481390 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] placement.version = None {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.481517 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.hw_machine_type = None {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.481634 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] quota.cores = 20 {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.481634 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] quota.count_usage_from_placement = False {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.481870 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_ceph_conf = {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.482007 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] quota.driver = nova.quota.DbQuotaDriver {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.482145 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_copy_poll_interval = 15 {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.482353 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_file_content_bytes = 10240 {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.482483 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_copy_timeout = 600 {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.482910 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_file_path_length = 255 {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.483059 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_store_name = {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.483465 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_files = 5 {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.483599 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_pool = rbd {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.484040 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] quota.instances = 10 {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.484189 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_type = default {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.484610 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] quota.key_pairs = 100 {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.484760 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_volume_group = None {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.485198 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] quota.metadata_items = 128 {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.485343 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_key = False {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.485879 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_partition = -2 {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.486008 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] quota.ram = 51200 {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.486463 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_password = False {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.486616 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] quota.recheck_quota = True {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.487036 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] quota.server_group_members = 10 {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.487163 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.iscsi_iface = None {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.487605 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] quota.server_groups = 10 {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.487780 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.iser_use_multipath = False {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.488182 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] rdp.enabled = False {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.488298 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_bandwidth = 0 {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.488747 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_completion_timeout = 800 {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.489175 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] rdp.html5_proxy_base_url = http://127.0.0.1:6083/ {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.489368 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime = 500 {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.489769 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.discover_hosts_in_cells_interval = -1 {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.489935 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime_delay = 75 {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.490330 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.enable_isolated_aggregate_filtering = False {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.490507 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime_steps = 10 {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.490888 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.image_metadata_prefilter = False {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.491075 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_inbound_addr = None {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.491453 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.limit_tenants_to_placement_aggregate = False {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.491643 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_permit_auto_converge = False {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.492014 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.max_attempts = 3 {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.492336 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_permit_post_copy = False {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.492888 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_scheme = None {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.493527 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_timeout_action = abort {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.493858 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.max_placement_results = 1000 {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.494101 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_tunnelled = False {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.494247 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.placement_aggregate_required_for_tenants = False {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.494663 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_uri = None {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.494809 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_availability_zone = True {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.495237 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_with_native_tls = False {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.495342 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_image_type_support = False {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.495801 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.max_queues = None {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.495914 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_routed_network_aggregates = False {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.496379 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.mem_stats_period_seconds = 10 {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.496527 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.workers = 2 {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.496791 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.aggregate_image_properties_isolation_namespace = None {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.496946 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.nfs_mount_options = None {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.497784 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.aggregate_image_properties_isolation_separator = . {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.497784 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.available_filters = ['nova.scheduler.filters.all_filters'] {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.498068 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.nfs_mount_point_base = /opt/stack/data/nova/mnt {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.498197 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.build_failure_weight_multiplier = 1000000.0 {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.498197 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.cpu_weight_multiplier = 1.0 {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.498458 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_aoe_discover_tries = 3 {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.498648 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.cross_cell_move_weight_multiplier = 1000000.0 {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.498871 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_iser_scan_tries = 5 {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.499017 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.disk_weight_multiplier = 1.0 {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.499387 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.enabled_filters = ['AvailabilityZoneFilter', 'ComputeFilter', 'ComputeCapabilitiesFilter', 'ImagePropertiesFilter', 'ServerGroupAntiAffinityFilter', 'ServerGroupAffinityFilter', 'SameHostFilter', 'DifferentHostFilter'] {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.499539 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_memory_encrypted_guests = None {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.499776 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.host_subset_size = 1 {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.500007 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_nvme_discover_tries = 5 {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.500176 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.image_properties_default_architecture = None {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.500541 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.io_ops_weight_multiplier = -1.0 {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.500694 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_pcie_ports = 0 {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.500860 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.isolated_hosts = [] {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.501256 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_volume_scan_tries = 5 {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.501401 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.isolated_images = [] {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.502920 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.pmem_namespaces = [] {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.503085 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.max_instances_per_host = 50 {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.503085 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.max_io_ops_per_host = 8 {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.503343 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.quobyte_client_cfg = None {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.503484 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.pci_weight_multiplier = 1.0 {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.503484 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.ram_weight_multiplier = 1.0 {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.504056 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.quobyte_mount_point_base = /opt/stack/data/nova/mnt {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.504924 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.restrict_isolated_hosts_to_isolated_images = True {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.505739 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_connect_timeout = 5 {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.506560 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.shuffle_best_same_weighed_hosts = False {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.506560 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.soft_affinity_weight_multiplier = 1.0 {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.507390 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_destroy_volume_retries = 12 {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.507544 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.soft_anti_affinity_weight_multiplier = 1.0 {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.507544 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.track_instance_changes = False {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.507842 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_destroy_volume_retry_interval = 5 {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.507987 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.weight_classes = ['nova.scheduler.weights.all_weighers'] {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.507987 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] metrics.required = True {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.508251 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_secret_uuid = None {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.508394 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_multiplier = 1.0 {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.508532 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_user = None {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.508674 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_of_unavailable = -10000.0 {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.508674 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_setting = [] {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.509255 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.realtime_scheduler_priority = 1 {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.509411 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.base_url = ws://127.0.0.1:6083/ {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.509553 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.remote_filesystem_transport = ssh {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.509704 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.enabled = False {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.509704 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.port_range = 10000:20000 {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.509981 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_image_id = None {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.510117 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.proxyclient_address = 127.0.0.1 {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.510117 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.serialproxy_host = 0.0.0.0 {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.510117 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.serialproxy_port = 6083 {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.510117 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] service_user.auth_section = None {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.510662 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_kernel_id = None {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.510836 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] service_user.auth_type = password {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.510836 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] service_user.cafile = None {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.511093 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_ramdisk_id = None {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.511245 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] service_user.certfile = None {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.511398 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rng_dev_path = /dev/urandom {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.511560 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] service_user.collect_timing = False {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.511560 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] service_user.insecure = False {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.511816 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rx_queue_size = None {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.511959 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] service_user.keyfile = None {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.511959 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] service_user.send_service_user_token = True {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.512226 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.smbfs_mount_options = {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.512350 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] service_user.split_loggers = False {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.512350 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] service_user.timeout = None {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.512594 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.smbfs_mount_point_base = /opt/stack/data/nova/mnt {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.512722 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] spice.agent_enabled = True {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.512722 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] spice.enabled = False {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.513024 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshot_compression = False {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.513560 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshot_image_format = None {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.513704 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_base_url = http://127.0.0.1:6082/spice_auto.html {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.514261 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshots_directory = /opt/stack/data/nova/instances/snapshots {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.514411 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_host = 0.0.0.0 {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.514834 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_port = 6082 {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.514983 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.sparse_logical_volumes = False {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.515398 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] spice.server_listen = 127.0.0.1 {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.515533 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_enabled = False {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.515962 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] spice.server_proxyclient_address = 127.0.0.1 {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.516119 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_group = tss {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.516511 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.baseapi = None {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.516636 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_user = tss {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.517042 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.cert = None {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.517243 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.sysinfo_serial = unique {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.517804 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.tx_queue_size = None {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.517928 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.compute = auto {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.518385 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.conductor = None {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.518508 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.uid_maps = [] {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.518926 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.scheduler = None {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.519050 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.use_virtio_for_bridges = True {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.519390 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.auth_section = None {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.519549 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.virt_type = kvm {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.519952 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.auth_type = None {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.520189 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_clear = zero {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.520463 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.cafile = None {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.520779 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_clear_size = 0 {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.520968 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.certfile = None {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.521394 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_use_multipath = False {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.521540 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.collect_timing = False {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.521924 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.insecure = False {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.522059 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_cache_path = None {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.522388 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.keyfile = None {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.522552 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_log_path = /var/log/vstorage/%(cluster_name)s/nova.log.gz {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.522950 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.split_loggers = False {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.523152 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_group = qemu {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.523492 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.timeout = None {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.523874 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_opts = [] {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.524068 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] vmware.api_retry_count = 10 {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.524407 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] vmware.ca_file = None {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.524550 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_perms = 0770 {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.524925 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] vmware.cache_prefix = None {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.525295 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_point_base = /opt/stack/data/nova/mnt {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.525470 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] vmware.cluster_name = None {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.526448 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_user = stack {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.526619 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] vmware.connection_pool_size = 10 {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.526619 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] vmware.console_delay_seconds = None {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.527011 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.wait_soft_reboot_seconds = 120 {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.527154 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] vmware.datastore_regex = None {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.527154 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_ip = None {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.527424 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] neutron.auth_section = None {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.527563 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_password = **** {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.527563 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_port = 443 {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.527837 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] neutron.auth_type = password {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.527987 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_username = None {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.528128 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] vmware.insecure = False {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.528279 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] neutron.cafile = None {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.528432 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] vmware.integration_bridge = None {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.528629 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] vmware.maximum_objects = 100 {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.528770 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] neutron.certfile = None {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.528910 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_default_policy = None {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.529257 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_enabled = False {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.529401 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] neutron.collect_timing = False {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.529616 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_wsdl_location = None {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.529877 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] neutron.connect_retries = None {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.530027 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_log_dir = /opt/vmware/vspc {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.530236 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_port_proxy_uri = None {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.530444 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] neutron.connect_retry_delay = None {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.530602 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_port_service_uri = None {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.530846 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] vmware.task_poll_interval = 0.5 {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.531042 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] neutron.default_floating_pool = public {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.531235 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] vmware.use_linked_clone = True {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.531464 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_keymap = en-us {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.531613 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] neutron.endpoint_override = None {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.531909 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_port = 5900 {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.532071 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_port_total = 10000 {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.532212 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] neutron.extension_sync_interval = 600 {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.532397 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] vnc.auth_schemes = ['none'] {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.532689 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] neutron.http_retries = 3 {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.532820 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] vnc.enabled = True {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.533300 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] neutron.insecure = False {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.533454 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_base_url = http://127.0.0.1:6080/vnc_auto.html {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.533623 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_host = 0.0.0.0 {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.533863 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] neutron.keyfile = None {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.534015 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_port = 6080 {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.534263 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] vnc.server_listen = 127.0.0.1 {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.534418 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] neutron.max_version = None {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.534577 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] vnc.server_proxyclient_address = 127.0.0.1 {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.534882 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_ca_certs = None {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.535083 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] neutron.metadata_proxy_shared_secret = **** {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.535227 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_client_cert = None {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.535477 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_client_key = None {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.535721 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] neutron.min_version = None {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.535858 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_compute_service_check_for_ffu = False {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.536095 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_fallback_pcpu_query = False {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.536227 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] neutron.ovs_bridge = br-int {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.536400 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_group_policy_check_upcall = False {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.536700 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] neutron.physnets = [] {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.536848 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_libvirt_livesnapshot = False {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.536987 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_rootwrap = False {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.537318 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] neutron.region_name = RegionOne {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.537459 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.enable_numa_live_migration = False {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.537622 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.enable_qemu_monitor_announce_self = False {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.537909 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_metadata_proxy = True {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.538040 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.ensure_libvirt_rbd_instance_dir_cleanup = False {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.538268 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.handle_virt_lifecycle_events = True {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.538473 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_name = None {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.538631 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.libvirt_disable_apic = False {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.538873 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.never_download_image_if_on_rbd = False {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.539144 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_type = network {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.539306 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.reserve_disk_resource_for_image_cache = False {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.539726 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] neutron.split_loggers = False {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.540315 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] neutron.status_code_retries = None {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.540898 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] neutron.status_code_retry_delay = None {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.541054 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.skip_cpu_compare_on_dest = False {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.541466 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.skip_hypervisor_version_check_on_lm = False {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.542463 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] neutron.timeout = None {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.542798 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.unified_limits_count_pcpu_as_vcpu = False {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.542798 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.wait_for_vif_plugged_event_during_hard_reboot = [] {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.542954 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] neutron.valid_interfaces = ['internal', 'public'] {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.543063 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.api_paste_config = /etc/nova/api-paste.ini {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.543063 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.client_socket_timeout = 900 {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.543339 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] neutron.version = None {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.543486 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.default_pool_size = 1000 {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.543644 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] notifications.bdms_in_notifications = False {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.543797 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.keep_alive = True {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.543797 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.max_header_line = 16384 {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.544046 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] notifications.default_level = INFO {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.544176 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.secure_proxy_ssl_header = None {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.544176 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_ca_file = None {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.544432 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] notifications.notification_format = unversioned {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.544577 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_cert_file = None {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.544740 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_key_file = None {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.545060 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.tcp_keepidle = 600 {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.545423 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.wsgi_log_format = %(client_ip)s "%(request_line)s" status: %(status_code)s len: %(body_length)s time: %(wall_seconds).7f {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.545780 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] notifications.notify_on_state_change = None {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.546418 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] notifications.versioned_notifications_topics = ['versioned_notifications'] {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.547037 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] pci.alias = [] {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.547551 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] zvm.ca_file = None {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.547715 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] pci.device_spec = [] {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.547910 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] zvm.cloud_connector_url = None {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.548208 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] placement.auth_section = None {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.548412 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] zvm.image_tmp_path = /opt/stack/data/nova/images {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.548926 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] zvm.reachable_timeout = 300 {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.549111 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] placement.auth_type = password {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.549316 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] oslo_versionedobjects.fatal_exception_format_errors = False {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.549497 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] placement.cafile = None {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.549630 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.549761 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.549935 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] placement.certfile = None {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.550080 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_process_name = {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.550325 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.550524 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] placement.collect_timing = False {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.550666 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.550948 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.enforce_new_defaults = False {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.551106 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] placement.connect_retries = None {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.551306 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.enforce_scope = False {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.551574 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_default_rule = default {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.551738 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] placement.connect_retry_delay = None {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.551916 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.552190 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] placement.endpoint_override = None {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.552348 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_file = policy.yaml {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.552553 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.552760 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] placement.insecure = False {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.552893 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.553187 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.553334 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] placement.keyfile = None {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.553472 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.553830 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.553954 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] placement.max_version = None {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.554153 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.auth_section = None {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.554439 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] placement.min_version = None {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.554583 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.auth_type = None {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.554909 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.cafile = None {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.555114 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] placement.region_name = RegionOne {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.555242 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.certfile = None {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.555557 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.collect_timing = False {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.555868 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.connect_retries = None {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.555978 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] placement.service_name = None {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.556168 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.connect_retry_delay = None {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.556471 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.endpoint_id = None {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.556616 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] placement.service_type = placement {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.557090 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.endpoint_override = None {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.557090 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.insecure = False {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.557413 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] placement.split_loggers = False {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.557545 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.keyfile = None {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.557671 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] placement.status_code_retries = None {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.557797 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.max_version = None {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.557970 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.min_version = None {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.558211 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] placement.status_code_retry_delay = None {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.558336 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.region_name = None {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.558559 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.service_name = None {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.558773 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] placement.timeout = None {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.558896 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.service_type = None {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.559152 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.split_loggers = False {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.559407 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] placement.valid_interfaces = ['internal', 'public'] {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.559541 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.status_code_retries = None {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.559740 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.status_code_retry_delay = None {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.559971 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] placement.version = None {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.560104 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.timeout = None {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.560326 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.valid_interfaces = None {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.560591 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] quota.cores = 20 {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.560713 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.version = None {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.560948 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.561189 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] quota.count_usage_from_placement = False {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.561313 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] profiler.connection_string = messaging:// {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.561618 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] profiler.enabled = False {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.561793 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] quota.driver = nova.quota.DbQuotaDriver {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.561938 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_doc_type = notification {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.562248 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_scroll_size = 10000 {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.562368 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_file_content_bytes = 10240 {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.562569 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_scroll_time = 2m {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.562875 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] profiler.filter_error_trace = False {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.563006 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_file_path_length = 255 {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.563194 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] profiler.hmac_keys = SECRET_KEY {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.563800 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_files = 5 {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.563992 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] profiler.sentinel_service_name = mymaster {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.564357 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] quota.instances = 10 {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.564502 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] profiler.socket_timeout = 0.1 {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.564662 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] profiler.trace_sqlalchemy = False {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.564944 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] quota.key_pairs = 100 {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.565081 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.amqp_auto_delete = False {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.565316 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.amqp_durable_queues = False {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.565559 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] quota.metadata_items = 128 {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.565696 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.conn_pool_min_size = 2 {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.565923 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.conn_pool_ttl = 1200 {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.566146 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] quota.ram = 51200 {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.566267 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.direct_mandatory_flag = True {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.566528 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.enable_cancel_on_failover = False {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.566720 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] quota.recheck_quota = True {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.566844 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_in_pthread = False {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.567146 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_rate = 3 {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.567304 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] quota.server_group_members = 10 {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.567466 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_timeout_threshold = 600 {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.567777 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_compression = None {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.567910 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] quota.server_groups = 10 {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.568101 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_failover_strategy = round-robin {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.568416 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_missing_consumer_retry_timeout = 60 {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.568539 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] rdp.enabled = False {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.568731 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_reconnect_delay = 1.0 {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.569043 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_ha_queues = False {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.569375 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_interval_max = 30 {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.569505 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] rdp.html5_proxy_base_url = http://127.0.0.1:6083/ {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.569704 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_login_method = AMQPLAIN {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.570011 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_qos_prefetch_count = 0 {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.570132 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.discover_hosts_in_cells_interval = -1 {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.570323 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quorum_delivery_limit = 0 {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.570693 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.enable_isolated_aggregate_filtering = False {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.570814 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quorum_queue = False {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.571279 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.image_metadata_prefilter = False {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.571530 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quroum_max_memory_bytes = 0 {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.571860 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.limit_tenants_to_placement_aggregate = False {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.572002 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quroum_max_memory_length = 0 {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.572257 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_retry_backoff = 2 {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.572437 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.max_attempts = 3 {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.572645 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_retry_interval = 1 {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.573016 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.max_placement_results = 1000 {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.573204 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_transient_queues_ttl = 1800 {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.573519 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rpc_conn_pool_size = 30 {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.573653 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.placement_aggregate_required_for_tenants = False {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.573842 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl = False {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.574175 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_availability_zone = True {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.574298 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_ca_file = {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.574503 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_cert_file = {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.574744 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_image_type_support = False {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.574868 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_enforce_fips_mode = False {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.575123 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_key_file = {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.575306 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_routed_network_aggregates = False {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.575452 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_version = {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.575808 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.driver = ['messagingv2'] {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.575935 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.workers = 2 {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.576117 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.retry = -1 {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.576461 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.topics = ['notifications'] {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.576583 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.aggregate_image_properties_isolation_namespace = None {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.576786 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.transport_url = **** {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.577056 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.aggregate_image_properties_isolation_separator = . {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.577204 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.file_event_handler = None {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.577422 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.file_event_handler_interval = 1 {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.577700 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.available_filters = ['nova.scheduler.filters.all_filters'] {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.577826 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.log_dir = None {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.577967 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi_app [-] ******************************************************************************** {{(pid=105769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Aug 27 02:13:27.578284 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.build_failure_weight_multiplier = 1000000.0 {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.578860 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.cpu_weight_multiplier = 1.0 {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.579437 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.cross_cell_move_weight_multiplier = 1000000.0 {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.580018 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.disk_weight_multiplier = 1.0 {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.580659 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.enabled_filters = ['AvailabilityZoneFilter', 'ComputeFilter', 'ComputeCapabilitiesFilter', 'ImagePropertiesFilter', 'ServerGroupAntiAffinityFilter', 'ServerGroupAffinityFilter', 'SameHostFilter', 'DifferentHostFilter'] {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.581273 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.host_subset_size = 1 {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.581845 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.image_properties_default_architecture = None {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.582416 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.io_ops_weight_multiplier = -1.0 {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.582997 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.isolated_hosts = [] {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.583573 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.isolated_images = [] {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.584154 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.max_instances_per_host = 50 {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.584732 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.max_io_ops_per_host = 8 {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.585337 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.pci_weight_multiplier = 1.0 {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.586044 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.ram_weight_multiplier = 1.0 {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.586568 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.restrict_isolated_hosts_to_isolated_images = True {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.587338 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.shuffle_best_same_weighed_hosts = False {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.587929 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.soft_affinity_weight_multiplier = 1.0 {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.588515 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.soft_anti_affinity_weight_multiplier = 1.0 {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.590533 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.track_instance_changes = False {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.590533 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.weight_classes = ['nova.scheduler.weights.all_weighers'] {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.590533 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] metrics.required = True {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.590944 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_multiplier = 1.0 {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.591477 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_of_unavailable = -10000.0 {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.592064 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_setting = [] {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.593074 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.base_url = ws://127.0.0.1:6083/ {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.593679 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.enabled = False {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.594293 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.port_range = 10000:20000 {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.594903 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.proxyclient_address = 127.0.0.1 {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.595512 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.serialproxy_host = 0.0.0.0 {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.596098 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.serialproxy_port = 6083 {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.596684 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] service_user.auth_section = None {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.597309 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] service_user.auth_type = password {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.597881 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] service_user.cafile = None {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.598450 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] service_user.certfile = None {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.601195 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] service_user.collect_timing = False {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.601195 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] service_user.insecure = False {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.601195 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] service_user.keyfile = None {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.601195 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] service_user.send_service_user_token = True {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.601782 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] service_user.split_loggers = False {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.602059 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] service_user.timeout = None {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.602834 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] spice.agent_enabled = True {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.603415 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] spice.enabled = False {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.604797 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_base_url = http://127.0.0.1:6082/spice_auto.html {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.605777 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_host = 0.0.0.0 {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.606050 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_port = 6082 {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.607030 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] spice.server_listen = 127.0.0.1 {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.607270 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] spice.server_proxyclient_address = 127.0.0.1 {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.607862 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.baseapi = None {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.608437 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.cert = None {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.609031 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.compute = auto {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.609633 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.conductor = None {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.610205 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.scheduler = None {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.610786 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.auth_section = None {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.611352 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.auth_type = None {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.611915 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.cafile = None {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.612540 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.certfile = None {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.613111 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.collect_timing = False {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.613711 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.insecure = False {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.614277 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.keyfile = None {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.614845 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.split_loggers = False {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.615405 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.timeout = None {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.616002 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] vmware.api_retry_count = 10 {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.616564 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] vmware.ca_file = None {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.617155 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] vmware.cache_prefix = None {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.617777 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] vmware.cluster_name = None {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.618355 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] vmware.connection_pool_size = 10 {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.618922 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] vmware.console_delay_seconds = None {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.619490 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] vmware.datastore_regex = None {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.620059 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_ip = None {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.620632 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_password = **** {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.621257 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_port = 443 {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.621821 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_username = None {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.622402 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] vmware.insecure = False {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.622973 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] vmware.integration_bridge = None {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.623552 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] vmware.maximum_objects = 100 {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.624122 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_default_policy = None {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.624695 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_enabled = False {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.625368 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_wsdl_location = None {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.625987 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_log_dir = /opt/vmware/vspc {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.626561 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_port_proxy_uri = None {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.627131 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_port_service_uri = None {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.627722 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] vmware.task_poll_interval = 0.5 {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.628302 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] vmware.use_linked_clone = True {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.628905 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_keymap = en-us {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.629522 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_port = 5900 {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.630106 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_port_total = 10000 {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.630762 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] vnc.auth_schemes = ['none'] {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.631355 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] vnc.enabled = True {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.632437 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_base_url = http://127.0.0.1:6080/vnc_auto.html {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.633100 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_host = 0.0.0.0 {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.633729 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_port = 6080 {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.634359 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] vnc.server_listen = 127.0.0.1 {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.634966 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] vnc.server_proxyclient_address = 127.0.0.1 {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.635546 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_ca_certs = None {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.636120 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_client_cert = None {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.636691 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_client_key = None {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.637335 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_compute_service_check_for_ffu = False {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.637917 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_fallback_pcpu_query = False {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.638491 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_group_policy_check_upcall = False {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.639070 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_libvirt_livesnapshot = False {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.639650 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_rootwrap = False {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.640227 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.enable_numa_live_migration = False {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.640809 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.enable_qemu_monitor_announce_self = False {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.641416 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.ensure_libvirt_rbd_instance_dir_cleanup = False {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.642122 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.handle_virt_lifecycle_events = True {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.642822 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.libvirt_disable_apic = False {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.643135 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.never_download_image_if_on_rbd = False {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.643710 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.reserve_disk_resource_for_image_cache = False {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.644283 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.skip_cpu_compare_on_dest = False {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.644856 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.skip_hypervisor_version_check_on_lm = False {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.645464 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.unified_limits_count_pcpu_as_vcpu = False {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.646050 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.wait_for_vif_plugged_event_during_hard_reboot = [] {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.646693 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.api_paste_config = /etc/nova/api-paste.ini {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.647277 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.client_socket_timeout = 900 {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.647861 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.default_pool_size = 1000 {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.648628 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.keep_alive = True {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.649278 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.max_header_line = 16384 {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.651070 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.secure_proxy_ssl_header = None {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.651070 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_ca_file = None {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.651070 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_cert_file = None {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.651626 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_key_file = None {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.651758 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.tcp_keepidle = 600 {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.652713 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.wsgi_log_format = %(client_ip)s "%(request_line)s" status: %(status_code)s len: %(body_length)s time: %(wall_seconds).7f {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.652713 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] zvm.ca_file = None {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.653166 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] zvm.cloud_connector_url = None {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.653815 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] zvm.image_tmp_path = /opt/stack/data/nova/images {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.654288 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] zvm.reachable_timeout = 300 {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.654775 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] oslo_versionedobjects.fatal_exception_format_errors = False {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.655263 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.655780 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.656283 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_process_name = {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.656775 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.657281 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.657753 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.enforce_new_defaults = False {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.658245 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.enforce_scope = False {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.658753 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_default_rule = default {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.659266 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.659792 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_file = policy.yaml {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.660263 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.660727 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.661310 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.661780 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.662248 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.662733 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.auth_section = None {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.663196 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.auth_type = None {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.663659 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.cafile = None {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.664132 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.certfile = None {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.664595 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.collect_timing = False {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.665057 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.connect_retries = None {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.665595 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.connect_retry_delay = None {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.666059 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.endpoint_id = None {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.666526 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.endpoint_override = None {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.666982 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.insecure = False {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.667444 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.keyfile = None {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.667904 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.max_version = None {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.668355 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.min_version = None {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.668820 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.region_name = None {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.669308 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.service_name = None {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.669772 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.service_type = None {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.670243 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.split_loggers = False {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.670696 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.status_code_retries = None {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.671160 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.status_code_retry_delay = None {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.671623 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.timeout = None {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.672087 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.valid_interfaces = None {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.672550 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.version = None {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.673069 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.673600 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] profiler.connection_string = messaging:// {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.674071 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] profiler.enabled = False {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.674552 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_doc_type = notification {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.675024 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_scroll_size = 10000 {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.675515 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_scroll_time = 2m {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.675979 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] profiler.filter_error_trace = False {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.676467 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] profiler.hmac_keys = SECRET_KEY {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.676960 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] profiler.sentinel_service_name = mymaster {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.677518 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] profiler.socket_timeout = 0.1 {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.677988 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] profiler.trace_sqlalchemy = False {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.678503 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.amqp_auto_delete = False {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.678979 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.amqp_durable_queues = False {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.679453 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.conn_pool_min_size = 2 {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.680292 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.conn_pool_ttl = 1200 {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.680761 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.direct_mandatory_flag = True {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.681268 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.enable_cancel_on_failover = False {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.681745 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_in_pthread = False {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.682243 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_rate = 3 {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.682725 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_timeout_threshold = 600 {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.683194 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_compression = None {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.683686 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_failover_strategy = round-robin {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.684176 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_missing_consumer_retry_timeout = 60 {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.684668 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_reconnect_delay = 1.0 {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.685180 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_ha_queues = False {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.685664 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_interval_max = 30 {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.686174 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_login_method = AMQPLAIN {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.686663 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_qos_prefetch_count = 0 {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.687141 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quorum_delivery_limit = 0 {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.687612 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quorum_queue = False {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.688098 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quroum_max_memory_bytes = 0 {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.688879 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quroum_max_memory_length = 0 {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.689393 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_retry_backoff = 2 {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.689873 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_retry_interval = 1 {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.690355 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_transient_queues_ttl = 1800 {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.690845 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rpc_conn_pool_size = 30 {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.691329 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl = False {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.691835 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_ca_file = {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.692333 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_cert_file = {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.692806 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_enforce_fips_mode = False {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.693337 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_key_file = {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.693836 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_version = {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.694361 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.driver = ['messagingv2'] {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.694840 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.retry = -1 {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.695368 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.topics = ['notifications'] {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.695861 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.transport_url = **** {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.696340 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.file_event_handler = None {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.696812 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.file_event_handler_interval = 1 {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.697302 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.log_dir = None {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:13:27.697664 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi_app [-] ******************************************************************************** {{(pid=105770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Aug 27 02:13:27.780679 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: INFO dbcounter [None req-fdc977b8-b815-4d55-885a-13acc1641c35 None None] Registered counter for database nova_api Aug 27 02:13:27.818678 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG oslo_db.sqlalchemy.engines [None req-fdc977b8-b815-4d55-885a-13acc1641c35 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=105769) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Aug 27 02:13:27.827362 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG dbcounter [-] [105769] Writer thread running {{(pid=105769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Aug 27 02:13:27.894655 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.context [None req-fdc977b8-b815-4d55-885a-13acc1641c35 None None] Found 2 cells: 00000000-0000-0000-0000-000000000000(cell0),41dc7c6b-010c-43dc-bd7e-0984fcde0c35(cell1) {{(pid=105769) load_cells /opt/stack/nova/nova/context.py:464}} Aug 27 02:13:27.898637 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG oslo_concurrency.lockutils [None req-fdc977b8-b815-4d55-885a-13acc1641c35 None None] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=105769) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:13:27.898921 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: INFO dbcounter [None req-c41012a0-e647-4622-a1e0-171ffd4c7276 None None] Registered counter for database nova_api Aug 27 02:13:27.899792 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG oslo_concurrency.lockutils [None req-fdc977b8-b815-4d55-885a-13acc1641c35 None None] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=105769) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:13:27.901456 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG oslo_concurrency.lockutils [None req-fdc977b8-b815-4d55-885a-13acc1641c35 None None] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=105769) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:13:27.902459 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG oslo_concurrency.lockutils [None req-fdc977b8-b815-4d55-885a-13acc1641c35 None None] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=105769) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:13:27.903183 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG oslo_concurrency.lockutils [None req-fdc977b8-b815-4d55-885a-13acc1641c35 None None] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=105769) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:13:27.906704 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG oslo_concurrency.lockutils [None req-fdc977b8-b815-4d55-885a-13acc1641c35 None None] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=105769) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:13:27.909784 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: INFO dbcounter [None req-fdc977b8-b815-4d55-885a-13acc1641c35 None None] Registered counter for database nova_cell0 Aug 27 02:13:27.917247 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: INFO dbcounter [None req-fdc977b8-b815-4d55-885a-13acc1641c35 None None] Registered counter for database nova_cell1 Aug 27 02:13:27.929110 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG oslo_db.sqlalchemy.engines [None req-fdc977b8-b815-4d55-885a-13acc1641c35 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=105769) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Aug 27 02:13:27.931383 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG oslo_db.sqlalchemy.engines [None req-c41012a0-e647-4622-a1e0-171ffd4c7276 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=105770) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Aug 27 02:13:27.932087 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG oslo_db.sqlalchemy.engines [None req-fdc977b8-b815-4d55-885a-13acc1641c35 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=105769) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Aug 27 02:13:27.936607 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG dbcounter [-] [105769] Writer thread running {{(pid=105769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Aug 27 02:13:27.940081 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG dbcounter [-] [105770] Writer thread running {{(pid=105770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Aug 27 02:13:28.010492 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.context [None req-c41012a0-e647-4622-a1e0-171ffd4c7276 None None] Found 2 cells: 00000000-0000-0000-0000-000000000000(cell0),41dc7c6b-010c-43dc-bd7e-0984fcde0c35(cell1) {{(pid=105770) load_cells /opt/stack/nova/nova/context.py:464}} Aug 27 02:13:28.019373 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG oslo_concurrency.lockutils [None req-c41012a0-e647-4622-a1e0-171ffd4c7276 None None] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=105770) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:13:28.020394 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG oslo_concurrency.lockutils [None req-c41012a0-e647-4622-a1e0-171ffd4c7276 None None] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=105770) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:13:28.023775 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG oslo_concurrency.lockutils [None req-c41012a0-e647-4622-a1e0-171ffd4c7276 None None] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=105770) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:13:28.025066 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG oslo_concurrency.lockutils [None req-c41012a0-e647-4622-a1e0-171ffd4c7276 None None] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=105770) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:13:28.025865 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG oslo_concurrency.lockutils [None req-c41012a0-e647-4622-a1e0-171ffd4c7276 None None] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=105770) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:13:28.029696 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG oslo_concurrency.lockutils [None req-c41012a0-e647-4622-a1e0-171ffd4c7276 None None] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=105770) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:13:28.033574 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: INFO dbcounter [None req-c41012a0-e647-4622-a1e0-171ffd4c7276 None None] Registered counter for database nova_cell0 Aug 27 02:13:28.043138 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: INFO dbcounter [None req-c41012a0-e647-4622-a1e0-171ffd4c7276 None None] Registered counter for database nova_cell1 Aug 27 02:13:28.055200 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG oslo_db.sqlalchemy.engines [None req-c41012a0-e647-4622-a1e0-171ffd4c7276 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=105770) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Aug 27 02:13:28.060346 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG oslo_db.sqlalchemy.engines [None req-c41012a0-e647-4622-a1e0-171ffd4c7276 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=105770) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Aug 27 02:13:28.071337 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG dbcounter [-] [105770] Writer thread running {{(pid=105770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Aug 27 02:13:28.073799 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG dbcounter [-] [105770] Writer thread running {{(pid=105770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Aug 27 02:13:28.117519 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG dbcounter [-] [105769] Writer thread running {{(pid=105769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Aug 27 02:13:28.132685 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: INFO dbcounter [None req-fdc977b8-b815-4d55-885a-13acc1641c35 None None] Registered counter for database nova_cell0 Aug 27 02:13:28.151437 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG oslo_db.sqlalchemy.engines [None req-fdc977b8-b815-4d55-885a-13acc1641c35 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=105769) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Aug 27 02:13:28.157764 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG dbcounter [-] [105769] Writer thread running {{(pid=105769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Aug 27 02:13:28.259610 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: INFO dbcounter [None req-c41012a0-e647-4622-a1e0-171ffd4c7276 None None] Registered counter for database nova_cell0 Aug 27 02:13:28.267140 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG oslo_db.sqlalchemy.engines [None req-c41012a0-e647-4622-a1e0-171ffd4c7276 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=105770) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Aug 27 02:13:28.272793 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG dbcounter [-] [105770] Writer thread running {{(pid=105770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Aug 27 02:13:28.553941 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: WARNING keystonemiddleware.auth_token [None req-fdc977b8-b815-4d55-885a-13acc1641c35 None None] AuthToken middleware is set with keystone_authtoken.service_token_roles_required set to False. This is backwards compatible but deprecated behaviour. Please set this to True. Aug 27 02:13:28.561633 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: WARNING keystonemiddleware.auth_token [None req-fdc977b8-b815-4d55-885a-13acc1641c35 None None] Configuring www_authenticate_uri to point to the public identity endpoint is required; clients may not be able to authenticate against an admin endpoint Aug 27 02:13:28.601756 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: WARNING keystonemiddleware.auth_token [None req-c41012a0-e647-4622-a1e0-171ffd4c7276 None None] AuthToken middleware is set with keystone_authtoken.service_token_roles_required set to False. This is backwards compatible but deprecated behaviour. Please set this to True. Aug 27 02:13:28.609490 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: WARNING keystonemiddleware.auth_token [None req-c41012a0-e647-4622-a1e0-171ffd4c7276 None None] Configuring www_authenticate_uri to point to the public identity endpoint is required; clients may not be able to authenticate against an admin endpoint Aug 27 02:13:28.973584 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: WARNING keystonemiddleware.auth_token [None req-fdc977b8-b815-4d55-885a-13acc1641c35 None None] AuthToken middleware is set with keystone_authtoken.service_token_roles_required set to False. This is backwards compatible but deprecated behaviour. Please set this to True. Aug 27 02:13:28.981618 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: WARNING keystonemiddleware.auth_token [None req-fdc977b8-b815-4d55-885a-13acc1641c35 None None] Configuring www_authenticate_uri to point to the public identity endpoint is required; clients may not be able to authenticate against an admin endpoint Aug 27 02:13:28.982343 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: WARNING keystonemiddleware.auth_token [None req-c41012a0-e647-4622-a1e0-171ffd4c7276 None None] AuthToken middleware is set with keystone_authtoken.service_token_roles_required set to False. This is backwards compatible but deprecated behaviour. Please set this to True. Aug 27 02:13:28.988656 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: WARNING keystonemiddleware.auth_token [None req-c41012a0-e647-4622-a1e0-171ffd4c7276 None None] Configuring www_authenticate_uri to point to the public identity endpoint is required; clients may not be able to authenticate against an admin endpoint Aug 27 02:13:28.993289 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x56112db8a9c0 pid: 105769 (default app) Aug 27 02:13:28.994359 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x56112db8a9c0 pid: 105770 (default app) Aug 27 02:13:28.996716 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi [None req-fdc977b8-b815-4d55-885a-13acc1641c35 None None] Calling method '>' {{(pid=105769) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:13:28.998573 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: INFO nova.api.openstack.requestlog [None req-fdc977b8-b815-4d55-885a-13acc1641c35 None None] 10.222.0.52 "GET /compute/v2.1/" status: 200 len: 388 microversion: 2.1 time: 0.002984 Aug 27 02:13:28.999437 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: [pid: 105769|app: 0|req: 1/1] 10.222.0.52 () {50 vars in 788 bytes} [Sat Aug 27 02:13:28 2022] GET /compute/v2.1/ => generated 388 bytes in 6 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 27 02:15:04.706842 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi [None req-c41012a0-e647-4622-a1e0-171ffd4c7276 None None] Calling method '>' {{(pid=105770) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:15:04.708987 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: INFO nova.api.openstack.requestlog [None req-c41012a0-e647-4622-a1e0-171ffd4c7276 None None] 10.222.0.52 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.003835 Aug 27 02:15:04.709442 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: [pid: 105770|app: 0|req: 1/2] 10.222.0.52 () {54 vars in 931 bytes} [Sat Aug 27 02:15:04 2022] GET /compute/v2.1 => generated 388 bytes in 6 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 27 02:15:04.725336 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG dbcounter [-] [105769] Writing DB stats nova_api:SELECT=3 {{(pid=105769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 27 02:15:04.727062 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG dbcounter [-] [105769] Writing DB stats nova_cell0:SELECT=3 {{(pid=105769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 27 02:15:04.729311 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG dbcounter [-] [105769] Writing DB stats nova_cell1:SELECT=3 {{(pid=105769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 27 02:15:04.732079 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG dbcounter [-] [105769] Writing DB stats nova_cell0:SELECT=6,nova_cell0:INSERT=1 {{(pid=105769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 27 02:15:05.699896 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi [None req-e1274b85-7cf3-42de-a78c-8278e6fbf034 admin admin] Calling method '>' {{(pid=105769) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:15:05.734545 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: INFO nova.api.openstack.requestlog [None req-e1274b85-7cf3-42de-a78c-8278e6fbf034 admin admin] 10.222.0.52 "GET /compute/v2.1/flavors/detail?is_public=True" status: 200 len: 15 microversion: 2.61 time: 1.015395 Aug 27 02:15:05.736027 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: [pid: 105769|app: 0|req: 2/3] 10.222.0.52 () {60 vars in 1292 bytes} [Sat Aug 27 02:15:04 2022] GET /compute/v2.1/flavors/detail?is_public=True => generated 15 bytes in 1020 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:15:08.417914 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi [None req-c41012a0-e647-4622-a1e0-171ffd4c7276 None None] Calling method '>' {{(pid=105770) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:15:08.418927 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: INFO nova.api.openstack.requestlog [None req-c41012a0-e647-4622-a1e0-171ffd4c7276 None None] 10.222.0.52 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001513 Aug 27 02:15:08.421482 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: [pid: 105770|app: 0|req: 2/4] 10.222.0.52 () {54 vars in 931 bytes} [Sat Aug 27 02:15:08 2022] GET /compute/v2.1 => generated 388 bytes in 5 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 27 02:15:08.992476 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi [None req-ca36afc5-873e-4302-8929-9cdbb7c90876 admin admin] Action: 'create', calling method: >, body: {"flavor": {"OS-FLV-EXT-DATA:ephemeral": 0, "id": "c1", "ram": 256, "rxtx_factor": 1.0, "vcpus": 1, "swap": 0, "os-flavor-access:is_public": true, "name": "cirros256", "disk": 1}} {{(pid=105769) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 27 02:15:09.044697 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: INFO nova.api.openstack.requestlog [None req-ca36afc5-873e-4302-8929-9cdbb7c90876 admin admin] 10.222.0.52 "POST /compute/v2.1/flavors" status: 200 len: 407 microversion: 2.61 time: 0.618260 Aug 27 02:15:09.045674 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: [pid: 105769|app: 0|req: 3/5] 10.222.0.52 () {64 vars in 1283 bytes} [Sat Aug 27 02:15:08 2022] POST /compute/v2.1/flavors => generated 407 bytes in 620 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Aug 27 02:15:09.060463 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG dbcounter [-] [105770] Writing DB stats nova_api:SELECT=3 {{(pid=105770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 27 02:15:09.061665 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG dbcounter [-] [105770] Writing DB stats nova_cell1:SELECT=3 {{(pid=105770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 27 02:15:09.062406 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG dbcounter [-] [105770] Writing DB stats nova_cell0:SELECT=3 {{(pid=105770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 27 02:15:09.063104 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG dbcounter [-] [105770] Writing DB stats nova_cell0:SELECT=3 {{(pid=105770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 27 02:15:09.286942 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi [None req-e00bbc0f-e275-451b-9874-31f2deaa62f8 admin admin] Action: 'create', calling method: >, body: {"extra_specs": {"hw_rng:allowed": "True"}} {{(pid=105770) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 27 02:15:09.358715 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: INFO nova.api.openstack.requestlog [None req-e00bbc0f-e275-451b-9874-31f2deaa62f8 admin admin] 10.222.0.52 "POST /compute/v2.1/flavors/c1/os-extra_specs" status: 200 len: 43 microversion: 2.61 time: 0.304693 Aug 27 02:15:09.359315 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: [pid: 105770|app: 0|req: 3/6] 10.222.0.52 () {64 vars in 1336 bytes} [Sat Aug 27 02:15:09 2022] POST /compute/v2.1/flavors/c1/os-extra_specs => generated 43 bytes in 306 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:15:11.804940 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi [None req-ca36afc5-873e-4302-8929-9cdbb7c90876 admin admin] Calling method '>' {{(pid=105769) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:15:11.805971 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: INFO nova.api.openstack.requestlog [None req-ca36afc5-873e-4302-8929-9cdbb7c90876 admin admin] 10.222.0.52 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001501 Aug 27 02:15:11.806658 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: [pid: 105769|app: 0|req: 4/7] 10.222.0.52 () {54 vars in 931 bytes} [Sat Aug 27 02:15:11 2022] GET /compute/v2.1 => generated 388 bytes in 3 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 27 02:15:12.553761 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi [None req-45e22361-0977-4a63-8e40-f7901a2a8fcf admin admin] Action: 'create', calling method: >, body: {"flavor": {"disk": 5, "id": "d1", "os-flavor-access:is_public": true, "ram": 512, "rxtx_factor": 1.0, "vcpus": 1, "swap": 0, "OS-FLV-EXT-DATA:ephemeral": 0, "name": "ds512M"}} {{(pid=105770) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 27 02:15:12.597393 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: INFO nova.api.openstack.requestlog [None req-45e22361-0977-4a63-8e40-f7901a2a8fcf admin admin] 10.222.0.52 "POST /compute/v2.1/flavors" status: 200 len: 404 microversion: 2.61 time: 0.784418 Aug 27 02:15:12.598746 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: [pid: 105770|app: 0|req: 4/8] 10.222.0.52 () {64 vars in 1283 bytes} [Sat Aug 27 02:15:11 2022] POST /compute/v2.1/flavors => generated 404 bytes in 786 msecs (HTTP/1.1 200) 9 headers in 359 bytes (2 switches on core 0) Aug 27 02:15:12.611432 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi [None req-a2d2a4cc-ee80-4274-8730-a42d5d9107fb admin admin] Action: 'create', calling method: >, body: {"extra_specs": {"hw_rng:allowed": "True"}} {{(pid=105769) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 27 02:15:12.655520 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: INFO nova.api.openstack.requestlog [None req-a2d2a4cc-ee80-4274-8730-a42d5d9107fb admin admin] 10.222.0.52 "POST /compute/v2.1/flavors/d1/os-extra_specs" status: 200 len: 43 microversion: 2.61 time: 0.047823 Aug 27 02:15:12.661932 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: [pid: 105769|app: 0|req: 5/9] 10.222.0.52 () {64 vars in 1336 bytes} [Sat Aug 27 02:15:12 2022] POST /compute/v2.1/flavors/d1/os-extra_specs => generated 43 bytes in 55 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 27 02:15:15.211693 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi [None req-45e22361-0977-4a63-8e40-f7901a2a8fcf admin admin] Calling method '>' {{(pid=105770) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:15:15.213060 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: INFO nova.api.openstack.requestlog [None req-45e22361-0977-4a63-8e40-f7901a2a8fcf admin admin] 10.222.0.52 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001468 Aug 27 02:15:15.213814 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: [pid: 105770|app: 0|req: 5/10] 10.222.0.52 () {54 vars in 931 bytes} [Sat Aug 27 02:15:15 2022] GET /compute/v2.1 => generated 388 bytes in 3 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 27 02:15:15.672096 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi [None req-21b59cb3-d0a4-494c-9a86-0d44179ebf30 admin admin] Action: 'create', calling method: >, body: {"flavor": {"ram": 1024, "rxtx_factor": 1.0, "os-flavor-access:is_public": true, "disk": 10, "name": "ds1G", "swap": 0, "OS-FLV-EXT-DATA:ephemeral": 0, "vcpus": 1, "id": "d2"}} {{(pid=105769) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 27 02:15:15.703788 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: INFO nova.api.openstack.requestlog [None req-21b59cb3-d0a4-494c-9a86-0d44179ebf30 admin admin] 10.222.0.52 "POST /compute/v2.1/flavors" status: 200 len: 404 microversion: 2.61 time: 0.483489 Aug 27 02:15:15.704336 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: [pid: 105769|app: 0|req: 6/11] 10.222.0.52 () {64 vars in 1283 bytes} [Sat Aug 27 02:15:15 2022] POST /compute/v2.1/flavors => generated 404 bytes in 485 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Aug 27 02:15:15.720202 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi [None req-49791991-f627-4eb1-9c8e-90c8fa2570e7 admin admin] Action: 'create', calling method: >, body: {"extra_specs": {"hw_rng:allowed": "True"}} {{(pid=105770) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 27 02:15:15.749953 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: INFO nova.api.openstack.requestlog [None req-49791991-f627-4eb1-9c8e-90c8fa2570e7 admin admin] 10.222.0.52 "POST /compute/v2.1/flavors/d2/os-extra_specs" status: 200 len: 43 microversion: 2.61 time: 0.033538 Aug 27 02:15:15.753473 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: [pid: 105770|app: 0|req: 6/12] 10.222.0.52 () {64 vars in 1336 bytes} [Sat Aug 27 02:15:15 2022] POST /compute/v2.1/flavors/d2/os-extra_specs => generated 43 bytes in 38 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 27 02:15:18.585492 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi [None req-21b59cb3-d0a4-494c-9a86-0d44179ebf30 admin admin] Calling method '>' {{(pid=105769) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:15:18.586853 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: INFO nova.api.openstack.requestlog [None req-21b59cb3-d0a4-494c-9a86-0d44179ebf30 admin admin] 10.222.0.52 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001704 Aug 27 02:15:18.587322 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: [pid: 105769|app: 0|req: 7/13] 10.222.0.52 () {54 vars in 931 bytes} [Sat Aug 27 02:15:18 2022] GET /compute/v2.1 => generated 388 bytes in 3 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 27 02:15:19.097094 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi [None req-a29ac5ab-782f-4de9-8ab6-38756b4fbda8 admin admin] Action: 'create', calling method: >, body: {"flavor": {"os-flavor-access:is_public": true, "disk": 10, "ram": 2048, "swap": 0, "id": "d3", "vcpus": 2, "OS-FLV-EXT-DATA:ephemeral": 0, "name": "ds2G", "rxtx_factor": 1.0}} {{(pid=105770) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 27 02:15:19.122202 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: INFO nova.api.openstack.requestlog [None req-a29ac5ab-782f-4de9-8ab6-38756b4fbda8 admin admin] 10.222.0.52 "POST /compute/v2.1/flavors" status: 200 len: 404 microversion: 2.61 time: 0.526807 Aug 27 02:15:19.130028 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: [pid: 105770|app: 0|req: 7/14] 10.222.0.52 () {64 vars in 1283 bytes} [Sat Aug 27 02:15:18 2022] POST /compute/v2.1/flavors => generated 404 bytes in 535 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Aug 27 02:15:19.135290 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi [None req-e0aa595b-e8f5-4975-a158-fcb95f6089c8 admin admin] Action: 'create', calling method: >, body: {"extra_specs": {"hw_rng:allowed": "True"}} {{(pid=105769) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 27 02:15:19.171579 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: INFO nova.api.openstack.requestlog [None req-e0aa595b-e8f5-4975-a158-fcb95f6089c8 admin admin] 10.222.0.52 "POST /compute/v2.1/flavors/d3/os-extra_specs" status: 200 len: 43 microversion: 2.61 time: 0.039989 Aug 27 02:15:19.186400 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: [pid: 105769|app: 0|req: 8/15] 10.222.0.52 () {64 vars in 1336 bytes} [Sat Aug 27 02:15:19 2022] POST /compute/v2.1/flavors/d3/os-extra_specs => generated 43 bytes in 55 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:15:21.661035 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi [None req-a29ac5ab-782f-4de9-8ab6-38756b4fbda8 admin admin] Calling method '>' {{(pid=105770) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:15:21.662808 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: INFO nova.api.openstack.requestlog [None req-a29ac5ab-782f-4de9-8ab6-38756b4fbda8 admin admin] 10.222.0.52 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.002218 Aug 27 02:15:21.665438 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: [pid: 105770|app: 0|req: 8/16] 10.222.0.52 () {54 vars in 931 bytes} [Sat Aug 27 02:15:21 2022] GET /compute/v2.1 => generated 388 bytes in 5 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Aug 27 02:15:22.149708 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi [None req-11c69916-470a-4ab9-90d0-46455d39f71e admin admin] Action: 'create', calling method: >, body: {"flavor": {"id": "d4", "os-flavor-access:is_public": true, "ram": 4096, "disk": 20, "swap": 0, "vcpus": 4, "name": "ds4G", "rxtx_factor": 1.0, "OS-FLV-EXT-DATA:ephemeral": 0}} {{(pid=105769) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 27 02:15:22.174310 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: INFO nova.api.openstack.requestlog [None req-11c69916-470a-4ab9-90d0-46455d39f71e admin admin] 10.222.0.52 "POST /compute/v2.1/flavors" status: 200 len: 404 microversion: 2.61 time: 0.504101 Aug 27 02:15:22.175226 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: [pid: 105769|app: 0|req: 9/17] 10.222.0.52 () {64 vars in 1283 bytes} [Sat Aug 27 02:15:21 2022] POST /compute/v2.1/flavors => generated 404 bytes in 506 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Aug 27 02:15:22.188270 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi [None req-c68643de-1f93-4f67-a795-bc2c94ae0957 admin admin] Action: 'create', calling method: >, body: {"extra_specs": {"hw_rng:allowed": "True"}} {{(pid=105770) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 27 02:15:22.219489 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: INFO nova.api.openstack.requestlog [None req-c68643de-1f93-4f67-a795-bc2c94ae0957 admin admin] 10.222.0.52 "POST /compute/v2.1/flavors/d4/os-extra_specs" status: 200 len: 43 microversion: 2.61 time: 0.034798 Aug 27 02:15:22.220218 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: [pid: 105770|app: 0|req: 9/18] 10.222.0.52 () {64 vars in 1336 bytes} [Sat Aug 27 02:15:22 2022] POST /compute/v2.1/flavors/d4/os-extra_specs => generated 43 bytes in 36 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:15:24.803834 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi [None req-11c69916-470a-4ab9-90d0-46455d39f71e admin admin] Calling method '>' {{(pid=105769) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:15:24.804715 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: INFO nova.api.openstack.requestlog [None req-11c69916-470a-4ab9-90d0-46455d39f71e admin admin] 10.222.0.52 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001493 Aug 27 02:15:24.805153 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: [pid: 105769|app: 0|req: 10/19] 10.222.0.52 () {54 vars in 931 bytes} [Sat Aug 27 02:15:24 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 27 02:15:25.381974 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi [None req-db912340-8f32-4e56-946e-78dc70b767cc admin admin] Calling method '>' {{(pid=105770) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:15:25.428253 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: INFO nova.api.openstack.requestlog [None req-db912340-8f32-4e56-946e-78dc70b767cc admin admin] 10.222.0.52 "GET /compute/v2.1/flavors/detail?is_public=True" status: 200 len: 2106 microversion: 2.61 time: 0.614453 Aug 27 02:15:25.429263 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: [pid: 105770|app: 0|req: 10/20] 10.222.0.52 () {60 vars in 1292 bytes} [Sat Aug 27 02:15:24 2022] GET /compute/v2.1/flavors/detail?is_public=True => generated 2106 bytes in 617 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 27 02:15:27.928166 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi [None req-11c69916-470a-4ab9-90d0-46455d39f71e admin admin] Calling method '>' {{(pid=105769) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:15:27.931018 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: INFO nova.api.openstack.requestlog [None req-11c69916-470a-4ab9-90d0-46455d39f71e admin admin] 10.222.0.52 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.002931 Aug 27 02:15:27.931932 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: [pid: 105769|app: 0|req: 11/21] 10.222.0.52 () {54 vars in 931 bytes} [Sat Aug 27 02:15:27 2022] GET /compute/v2.1 => generated 388 bytes in 5 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 27 02:15:28.406006 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi [None req-df7cb7b5-1725-426d-85a7-6831be474168 admin admin] Action: 'create', calling method: >, body: {"flavor": {"OS-FLV-EXT-DATA:ephemeral": 0, "id": "1", "swap": 0, "os-flavor-access:is_public": true, "rxtx_factor": 1.0, "ram": 512, "name": "m1.tiny", "disk": 1, "vcpus": 1}} {{(pid=105770) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 27 02:15:28.429888 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: INFO nova.api.openstack.requestlog [None req-df7cb7b5-1725-426d-85a7-6831be474168 admin admin] 10.222.0.52 "POST /compute/v2.1/flavors" status: 200 len: 402 microversion: 2.61 time: 0.490518 Aug 27 02:15:28.430707 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: [pid: 105770|app: 0|req: 11/22] 10.222.0.52 () {64 vars in 1283 bytes} [Sat Aug 27 02:15:27 2022] POST /compute/v2.1/flavors => generated 402 bytes in 492 msecs (HTTP/1.1 200) 9 headers in 359 bytes (2 switches on core 0) Aug 27 02:15:28.443789 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi [None req-3100e704-d074-46ab-8fc1-b2d369b43629 admin admin] Action: 'create', calling method: >, body: {"extra_specs": {"hw_rng:allowed": "True"}} {{(pid=105769) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 27 02:15:28.479773 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: INFO nova.api.openstack.requestlog [None req-3100e704-d074-46ab-8fc1-b2d369b43629 admin admin] 10.222.0.52 "POST /compute/v2.1/flavors/1/os-extra_specs" status: 200 len: 43 microversion: 2.61 time: 0.039971 Aug 27 02:15:28.480369 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: [pid: 105769|app: 0|req: 12/23] 10.222.0.52 () {64 vars in 1333 bytes} [Sat Aug 27 02:15:28 2022] POST /compute/v2.1/flavors/1/os-extra_specs => generated 43 bytes in 41 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:15:30.946377 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi [None req-df7cb7b5-1725-426d-85a7-6831be474168 admin admin] Calling method '>' {{(pid=105770) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:15:30.948853 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: INFO nova.api.openstack.requestlog [None req-df7cb7b5-1725-426d-85a7-6831be474168 admin admin] 10.222.0.52 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.003160 Aug 27 02:15:30.949707 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: [pid: 105770|app: 0|req: 12/24] 10.222.0.52 () {54 vars in 931 bytes} [Sat Aug 27 02:15:30 2022] GET /compute/v2.1 => generated 388 bytes in 5 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 27 02:15:31.390713 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi [None req-19b17199-6ee5-4a42-bc4c-143cb7ef8d03 admin admin] Action: 'create', calling method: >, body: {"flavor": {"name": "m1.small", "os-flavor-access:is_public": true, "disk": 20, "swap": 0, "rxtx_factor": 1.0, "ram": 2048, "id": "2", "vcpus": 1, "OS-FLV-EXT-DATA:ephemeral": 0}} {{(pid=105769) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 27 02:15:31.411329 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: INFO nova.api.openstack.requestlog [None req-19b17199-6ee5-4a42-bc4c-143cb7ef8d03 admin admin] 10.222.0.52 "POST /compute/v2.1/flavors" status: 200 len: 405 microversion: 2.61 time: 0.455122 Aug 27 02:15:31.411888 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: [pid: 105769|app: 0|req: 13/25] 10.222.0.52 () {64 vars in 1283 bytes} [Sat Aug 27 02:15:30 2022] POST /compute/v2.1/flavors => generated 405 bytes in 456 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Aug 27 02:15:31.424845 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi [None req-d48c9345-0588-45cf-b757-23f891fef10f admin admin] Action: 'create', calling method: >, body: {"extra_specs": {"hw_rng:allowed": "True"}} {{(pid=105770) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 27 02:15:31.453690 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: INFO nova.api.openstack.requestlog [None req-d48c9345-0588-45cf-b757-23f891fef10f admin admin] 10.222.0.52 "POST /compute/v2.1/flavors/2/os-extra_specs" status: 200 len: 43 microversion: 2.61 time: 0.032465 Aug 27 02:15:31.454530 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: [pid: 105770|app: 0|req: 13/26] 10.222.0.52 () {64 vars in 1333 bytes} [Sat Aug 27 02:15:31 2022] POST /compute/v2.1/flavors/2/os-extra_specs => generated 43 bytes in 34 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 27 02:15:33.965893 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi [None req-19b17199-6ee5-4a42-bc4c-143cb7ef8d03 admin admin] Calling method '>' {{(pid=105769) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:15:33.967705 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: INFO nova.api.openstack.requestlog [None req-19b17199-6ee5-4a42-bc4c-143cb7ef8d03 admin admin] 10.222.0.52 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.002303 Aug 27 02:15:33.968217 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: [pid: 105769|app: 0|req: 14/27] 10.222.0.52 () {54 vars in 931 bytes} [Sat Aug 27 02:15:33 2022] GET /compute/v2.1 => generated 388 bytes in 3 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 27 02:15:34.543722 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi [None req-e1e01d73-8ef3-455c-a449-6773e48b06ad admin admin] Action: 'create', calling method: >, body: {"flavor": {"swap": 0, "name": "m1.medium", "ram": 4096, "os-flavor-access:is_public": true, "OS-FLV-EXT-DATA:ephemeral": 0, "vcpus": 2, "disk": 40, "rxtx_factor": 1.0, "id": "3"}} {{(pid=105770) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 27 02:15:34.563636 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: INFO nova.api.openstack.requestlog [None req-e1e01d73-8ef3-455c-a449-6773e48b06ad admin admin] 10.222.0.52 "POST /compute/v2.1/flavors" status: 200 len: 406 microversion: 2.61 time: 0.588915 Aug 27 02:15:34.564181 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: [pid: 105770|app: 0|req: 14/28] 10.222.0.52 () {64 vars in 1283 bytes} [Sat Aug 27 02:15:33 2022] POST /compute/v2.1/flavors => generated 406 bytes in 590 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Aug 27 02:15:34.576686 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi [None req-15bc0d2d-e11b-4d19-a04e-ef0e3a92471d admin admin] Action: 'create', calling method: >, body: {"extra_specs": {"hw_rng:allowed": "True"}} {{(pid=105769) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 27 02:15:34.601351 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: INFO nova.api.openstack.requestlog [None req-15bc0d2d-e11b-4d19-a04e-ef0e3a92471d admin admin] 10.222.0.52 "POST /compute/v2.1/flavors/3/os-extra_specs" status: 200 len: 43 microversion: 2.61 time: 0.028355 Aug 27 02:15:34.617149 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: [pid: 105769|app: 0|req: 15/29] 10.222.0.52 () {64 vars in 1333 bytes} [Sat Aug 27 02:15:34 2022] POST /compute/v2.1/flavors/3/os-extra_specs => generated 43 bytes in 37 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:15:37.143837 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi [None req-e1e01d73-8ef3-455c-a449-6773e48b06ad admin admin] Calling method '>' {{(pid=105770) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:15:37.145820 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: INFO nova.api.openstack.requestlog [None req-e1e01d73-8ef3-455c-a449-6773e48b06ad admin admin] 10.222.0.52 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.002713 Aug 27 02:15:37.147001 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: [pid: 105770|app: 0|req: 15/30] 10.222.0.52 () {54 vars in 931 bytes} [Sat Aug 27 02:15:37 2022] GET /compute/v2.1 => generated 388 bytes in 4 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 27 02:15:37.733865 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi [None req-bfcac5e9-0c6c-4f04-9f62-3e8899793925 admin admin] Action: 'create', calling method: >, body: {"flavor": {"disk": 80, "id": "4", "ram": 8192, "swap": 0, "OS-FLV-EXT-DATA:ephemeral": 0, "os-flavor-access:is_public": true, "vcpus": 4, "rxtx_factor": 1.0, "name": "m1.large"}} {{(pid=105769) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 27 02:15:37.742073 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG dbcounter [-] [105769] Writing DB stats nova_api:SELECT=39,nova_api:INSERT=8,nova_api:SAVEPOINT=4,nova_api:RELEASE=4 {{(pid=105769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 27 02:15:37.754580 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: INFO nova.api.openstack.requestlog [None req-bfcac5e9-0c6c-4f04-9f62-3e8899793925 admin admin] 10.222.0.52 "POST /compute/v2.1/flavors" status: 200 len: 405 microversion: 2.61 time: 0.593097 Aug 27 02:15:37.755416 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: [pid: 105769|app: 0|req: 16/31] 10.222.0.52 () {64 vars in 1283 bytes} [Sat Aug 27 02:15:37 2022] POST /compute/v2.1/flavors => generated 405 bytes in 594 msecs (HTTP/1.1 200) 9 headers in 359 bytes (2 switches on core 0) Aug 27 02:15:37.769167 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi [None req-6f216799-273c-4bdb-a9d5-911280da9852 admin admin] Action: 'create', calling method: >, body: {"extra_specs": {"hw_rng:allowed": "True"}} {{(pid=105770) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 27 02:15:37.814393 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: INFO nova.api.openstack.requestlog [None req-6f216799-273c-4bdb-a9d5-911280da9852 admin admin] 10.222.0.52 "POST /compute/v2.1/flavors/4/os-extra_specs" status: 200 len: 43 microversion: 2.61 time: 0.049043 Aug 27 02:15:37.815709 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: [pid: 105770|app: 0|req: 16/32] 10.222.0.52 () {64 vars in 1333 bytes} [Sat Aug 27 02:15:37 2022] POST /compute/v2.1/flavors/4/os-extra_specs => generated 43 bytes in 52 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 27 02:15:40.562614 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi [None req-bfcac5e9-0c6c-4f04-9f62-3e8899793925 admin admin] Calling method '>' {{(pid=105769) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:15:40.563585 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: INFO nova.api.openstack.requestlog [None req-bfcac5e9-0c6c-4f04-9f62-3e8899793925 admin admin] 10.222.0.52 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001487 Aug 27 02:15:40.564204 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: [pid: 105769|app: 0|req: 17/33] 10.222.0.52 () {54 vars in 931 bytes} [Sat Aug 27 02:15:40 2022] GET /compute/v2.1 => generated 388 bytes in 3 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 27 02:15:41.357436 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi [None req-405dad5a-527b-4870-926e-747887ddb00c admin admin] Action: 'create', calling method: >, body: {"flavor": {"rxtx_factor": 1.0, "vcpus": 8, "OS-FLV-EXT-DATA:ephemeral": 0, "id": "5", "os-flavor-access:is_public": true, "name": "m1.xlarge", "ram": 16384, "swap": 0, "disk": 160}} {{(pid=105770) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 27 02:15:41.363362 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG dbcounter [-] [105770] Writing DB stats nova_api:SELECT=45,nova_api:SAVEPOINT=5,nova_api:INSERT=9,nova_api:RELEASE=5 {{(pid=105770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 27 02:15:41.382142 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: INFO nova.api.openstack.requestlog [None req-405dad5a-527b-4870-926e-747887ddb00c admin admin] 10.222.0.52 "POST /compute/v2.1/flavors" status: 200 len: 408 microversion: 2.61 time: 0.805593 Aug 27 02:15:41.383055 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: [pid: 105770|app: 0|req: 17/34] 10.222.0.52 () {64 vars in 1283 bytes} [Sat Aug 27 02:15:40 2022] POST /compute/v2.1/flavors => generated 408 bytes in 807 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Aug 27 02:15:41.398538 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi [None req-ed856794-0155-4c9a-bc53-57fa20e7dec8 admin admin] Action: 'create', calling method: >, body: {"extra_specs": {"hw_rng:allowed": "True"}} {{(pid=105769) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 27 02:15:41.424624 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: INFO nova.api.openstack.requestlog [None req-ed856794-0155-4c9a-bc53-57fa20e7dec8 admin admin] 10.222.0.52 "POST /compute/v2.1/flavors/5/os-extra_specs" status: 200 len: 43 microversion: 2.61 time: 0.032619 Aug 27 02:15:41.425424 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: [pid: 105769|app: 0|req: 18/35] 10.222.0.52 () {64 vars in 1333 bytes} [Sat Aug 27 02:15:41 2022] POST /compute/v2.1/flavors/5/os-extra_specs => generated 43 bytes in 34 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:15:59.519026 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi [None req-405dad5a-527b-4870-926e-747887ddb00c admin admin] Calling method '>' {{(pid=105770) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:15:59.521781 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: INFO nova.api.openstack.requestlog [None req-405dad5a-527b-4870-926e-747887ddb00c admin admin] 10.222.0.52 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001488 Aug 27 02:15:59.521781 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: [pid: 105770|app: 0|req: 18/36] 10.222.0.52 () {54 vars in 931 bytes} [Sat Aug 27 02:15:59 2022] GET /compute/v2.1 => generated 388 bytes in 3 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 27 02:15:59.531947 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG dbcounter [-] [105769] Writing DB stats nova_api:INSERT=2,nova_api:SELECT=8,nova_api:SAVEPOINT=1,nova_api:RELEASE=1 {{(pid=105769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 27 02:16:00.000946 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi [None req-891e1c1c-1c9e-42af-bc23-0500f15c872a admin admin] Calling method '>' {{(pid=105769) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:16:00.020166 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: INFO nova.api.openstack.requestlog [None req-891e1c1c-1c9e-42af-bc23-0500f15c872a admin admin] 10.222.0.52 "GET /compute/v2.1/flavors/detail?is_public=True" status: 200 len: 4202 microversion: 2.61 time: 0.492565 Aug 27 02:16:00.020722 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: [pid: 105769|app: 0|req: 19/37] 10.222.0.52 () {60 vars in 1292 bytes} [Sat Aug 27 02:15:59 2022] GET /compute/v2.1/flavors/detail?is_public=True => generated 4202 bytes in 494 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 27 02:16:01.160717 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi [None req-405dad5a-527b-4870-926e-747887ddb00c admin admin] Calling method '>' {{(pid=105770) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:16:01.161735 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: INFO nova.api.openstack.requestlog [None req-405dad5a-527b-4870-926e-747887ddb00c admin admin] 10.222.0.52 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001482 Aug 27 02:16:01.162147 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: [pid: 105770|app: 0|req: 19/38] 10.222.0.52 () {56 vars in 1074 bytes} [Sat Aug 27 02:16:01 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 27 02:16:01.765973 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi [None req-9a730d2a-c71b-49fb-8042-347dbdb5de6c admin admin] Calling method '>' {{(pid=105769) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:16:01.772291 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: INFO nova.api.openstack.wsgi [None req-9a730d2a-c71b-49fb-8042-347dbdb5de6c admin admin] HTTP exception thrown: Flavor 42 could not be found. Aug 27 02:16:01.775664 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi [None req-9a730d2a-c71b-49fb-8042-347dbdb5de6c admin admin] Request method failure captured: Aug 27 02:16:01.775664 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: request: GET /compute/v2.1/flavors/42 HTTP/1.1 Aug 27 02:16:01.775664 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: Accept: application/json Aug 27 02:16:01.775664 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: Accept-Encoding: gzip, deflate Aug 27 02:16:01.775664 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: Connection: keep-alive Aug 27 02:16:01.775664 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: Content-Length: 0 Aug 27 02:16:01.775664 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: Host: 10.222.0.52 Aug 27 02:16:01.775664 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: Openstack-Api-Version: compute 2.90 Aug 27 02:16:01.775664 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: Openstack-System-Scope: None Aug 27 02:16:01.775664 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: User-Agent: python-novaclient Aug 27 02:16:01.775664 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: X-Auth-Token: gAAAAABjCX5hL79eXDp994PKZMMHjTsFVprJxHHb-4N2stuy4VQYQjtMTCzzgl78L7fAqzrdUCNl0BVbkDmSobGsqCFsKWYEkh9MACX95cRtwz184cokxSB605KeyYEIq4hZtVmZVLQthb-RKwIPqco_O5jZeYOTjnl2pTuWItYRw0pg-Hn9rmw Aug 27 02:16:01.775664 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: X-Domain-Id: None Aug 27 02:16:01.775664 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: X-Domain-Name: None Aug 27 02:16:01.775664 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: X-Identity-Status: Confirmed Aug 27 02:16:01.775664 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: X-Is-Admin-Project: True Aug 27 02:16:01.775664 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: X-Openstack-Nova-Api-Version: 2.90 Aug 27 02:16:01.775664 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: X-Project-Domain-Id: default Aug 27 02:16:01.775664 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: X-Project-Domain-Name: Default Aug 27 02:16:01.775664 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: X-Project-Id: f1df9f889b3642a7ac056884bc2c51a4 Aug 27 02:16:01.775664 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: X-Project-Name: admin Aug 27 02:16:01.775664 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: X-Role: reader,admin,member Aug 27 02:16:01.775664 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: X-Roles: reader,admin,member Aug 27 02:16:01.778028 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: X-Service-Catalog: [{"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.52:8080", "publicURL": "http://10.222.0.52:8080/v1/AUTH_f1df9f889b3642a7ac056884bc2c51a4"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52:9696/networking"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/compute/v2/f1df9f889b3642a7ac056884bc2c51a4"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/image"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/volume/v3/f1df9f889b3642a7ac056884bc2c51a4"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/identity"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/volume/v3/f1df9f889b3642a7ac056884bc2c51a4"}]}] Aug 27 02:16:01.778028 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: X-Tenant: admin Aug 27 02:16:01.778028 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: X-Tenant-Id: f1df9f889b3642a7ac056884bc2c51a4 Aug 27 02:16:01.778028 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: X-Tenant-Name: admin Aug 27 02:16:01.778028 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: X-User: admin Aug 27 02:16:01.778028 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: X-User-Domain-Id: default Aug 27 02:16:01.778028 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: X-User-Domain-Name: Default Aug 27 02:16:01.778028 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: X-User-Id: 6648709a4b6d470c80e9b42cb0d345ad Aug 27 02:16:01.778028 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: X-User-Name: admin Aug 27 02:16:01.778028 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: method: > Aug 27 02:16:01.778028 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: exception: Flavor 42 could not be found. Aug 27 02:16:01.778028 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: action_args: {'id': '42'} Aug 27 02:16:01.778028 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: {{(pid=105769) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Aug 27 02:16:01.778028 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 27 02:16:01.780134 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 66, in show Aug 27 02:16:01.780134 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: ERROR nova.api.openstack.wsgi flavor = flavors.get_flavor_by_flavor_id(id, ctxt=context) Aug 27 02:16:01.780134 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/flavors.py", line 139, in get_flavor_by_flavor_id Aug 27 02:16:01.780134 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: ERROR nova.api.openstack.wsgi return objects.Flavor.get_by_flavor_id(ctxt, flavorid, read_deleted) Aug 27 02:16:01.780134 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_versionedobjects/base.py", line 184, in wrapper Aug 27 02:16:01.780134 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: ERROR nova.api.openstack.wsgi result = fn(cls, context, *args, **kwargs) Aug 27 02:16:01.780134 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 390, in get_by_flavor_id Aug 27 02:16:01.780134 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: ERROR nova.api.openstack.wsgi db_flavor = cls._flavor_get_by_flavor_id_from_db(context, Aug 27 02:16:01.780134 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/db/utils.py", line 35, in wrapper Aug 27 02:16:01.780134 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Aug 27 02:16:01.780134 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 317, in _flavor_get_by_flavor_id_from_db Aug 27 02:16:01.780134 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: ERROR nova.api.openstack.wsgi raise exception.FlavorNotFound(flavor_id=flavor_id) Aug 27 02:16:01.780134 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: ERROR nova.api.openstack.wsgi nova.exception.FlavorNotFound: Flavor 42 could not be found. Aug 27 02:16:01.780134 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: ERROR nova.api.openstack.wsgi Aug 27 02:16:01.780134 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 27 02:16:01.780134 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: ERROR nova.api.openstack.wsgi Aug 27 02:16:01.780134 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 27 02:16:01.780134 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 27 02:16:01.780134 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 27 02:16:01.782409 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Aug 27 02:16:01.782409 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Aug 27 02:16:01.782409 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Aug 27 02:16:01.782409 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Aug 27 02:16:01.782409 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 68, in show Aug 27 02:16:01.782409 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPNotFound(explanation=e.format_message()) Aug 27 02:16:01.782409 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Flavor 42 could not be found. Aug 27 02:16:01.782409 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: ERROR nova.api.openstack.wsgi Aug 27 02:16:01.782409 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 27 02:16:01.782409 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: ERROR nova.api.openstack.wsgi Aug 27 02:16:01.782409 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 27 02:16:01.782409 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 27 02:16:01.782409 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 27 02:16:01.782409 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Aug 27 02:16:01.782409 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Aug 27 02:16:01.782409 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Flavor 42 could not be found. Aug 27 02:16:01.782409 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: ERROR nova.api.openstack.wsgi Aug 27 02:16:01.782409 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi [None req-9a730d2a-c71b-49fb-8042-347dbdb5de6c admin admin] Returning 404 to user: Flavor 42 could not be found. {{(pid=105769) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Aug 27 02:16:01.784588 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: INFO nova.api.openstack.requestlog [None req-9a730d2a-c71b-49fb-8042-347dbdb5de6c admin admin] 10.222.0.52 "GET /compute/v2.1/flavors/42" status: 404 len: 75 microversion: 2.90 time: 0.463504 Aug 27 02:16:01.784588 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: [pid: 105769|app: 0|req: 20/39] 10.222.0.52 () {60 vars in 1190 bytes} [Sat Aug 27 02:16:01 2022] GET /compute/v2.1/flavors/42 => generated 75 bytes in 465 msecs (HTTP/1.1 404) 9 headers in 380 bytes (1 switches on core 0) Aug 27 02:16:01.788952 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi [None req-faed0f12-b1ed-4935-9c92-346ddcdf83ba admin admin] Calling method '>' {{(pid=105770) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:16:01.814064 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: INFO nova.api.openstack.requestlog [None req-faed0f12-b1ed-4935-9c92-346ddcdf83ba admin admin] 10.222.0.52 "GET /compute/v2.1/flavors?is_public=None" status: 200 len: 2056 microversion: 2.90 time: 0.029748 Aug 27 02:16:01.814640 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: [pid: 105770|app: 0|req: 20/40] 10.222.0.52 () {60 vars in 1210 bytes} [Sat Aug 27 02:16:01 2022] GET /compute/v2.1/flavors?is_public=None => generated 2056 bytes in 31 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Aug 27 02:16:01.825711 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi [None req-cdebfe6c-729c-4762-9578-a75236ee093a admin admin] Calling method '>' {{(pid=105769) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:16:01.851467 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: INFO nova.api.openstack.requestlog [None req-cdebfe6c-729c-4762-9578-a75236ee093a admin admin] 10.222.0.52 "GET /compute/v2.1/flavors?is_public=None" status: 200 len: 2056 microversion: 2.90 time: 0.029556 Aug 27 02:16:01.852580 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: [pid: 105769|app: 0|req: 21/41] 10.222.0.52 () {60 vars in 1210 bytes} [Sat Aug 27 02:16:01 2022] GET /compute/v2.1/flavors?is_public=None => generated 2056 bytes in 31 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 27 02:16:01.862794 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi [None req-e18e551f-0027-461a-bd65-24d4f09fdaaa admin admin] Calling method '>' {{(pid=105770) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:16:01.868994 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: INFO nova.api.openstack.wsgi [None req-e18e551f-0027-461a-bd65-24d4f09fdaaa admin admin] HTTP exception thrown: Flavor 42 could not be found. Aug 27 02:16:01.873217 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi [None req-e18e551f-0027-461a-bd65-24d4f09fdaaa admin admin] Request method failure captured: Aug 27 02:16:01.873217 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: request: GET /compute/v2.1/flavors/42 HTTP/1.1 Aug 27 02:16:01.873217 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: Accept: application/json Aug 27 02:16:01.873217 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: Accept-Encoding: gzip, deflate Aug 27 02:16:01.873217 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: Connection: keep-alive Aug 27 02:16:01.873217 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: Content-Length: 0 Aug 27 02:16:01.873217 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: Host: 10.222.0.52 Aug 27 02:16:01.873217 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: Openstack-Api-Version: compute 2.90 Aug 27 02:16:01.873217 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: Openstack-System-Scope: None Aug 27 02:16:01.873217 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: User-Agent: python-novaclient Aug 27 02:16:01.873217 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: X-Auth-Token: gAAAAABjCX5hL79eXDp994PKZMMHjTsFVprJxHHb-4N2stuy4VQYQjtMTCzzgl78L7fAqzrdUCNl0BVbkDmSobGsqCFsKWYEkh9MACX95cRtwz184cokxSB605KeyYEIq4hZtVmZVLQthb-RKwIPqco_O5jZeYOTjnl2pTuWItYRw0pg-Hn9rmw Aug 27 02:16:01.873217 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: X-Domain-Id: None Aug 27 02:16:01.873217 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: X-Domain-Name: None Aug 27 02:16:01.873217 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: X-Identity-Status: Confirmed Aug 27 02:16:01.873217 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: X-Is-Admin-Project: True Aug 27 02:16:01.873217 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: X-Openstack-Nova-Api-Version: 2.90 Aug 27 02:16:01.873217 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: X-Project-Domain-Id: default Aug 27 02:16:01.873217 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: X-Project-Domain-Name: Default Aug 27 02:16:01.873217 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: X-Project-Id: f1df9f889b3642a7ac056884bc2c51a4 Aug 27 02:16:01.873217 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: X-Project-Name: admin Aug 27 02:16:01.873217 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: X-Role: reader,admin,member Aug 27 02:16:01.873217 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: X-Roles: reader,admin,member Aug 27 02:16:01.874785 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: X-Service-Catalog: [{"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.52:8080", "publicURL": "http://10.222.0.52:8080/v1/AUTH_f1df9f889b3642a7ac056884bc2c51a4"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52:9696/networking"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/compute/v2/f1df9f889b3642a7ac056884bc2c51a4"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/image"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/volume/v3/f1df9f889b3642a7ac056884bc2c51a4"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/identity"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/volume/v3/f1df9f889b3642a7ac056884bc2c51a4"}]}] Aug 27 02:16:01.874785 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: X-Tenant: admin Aug 27 02:16:01.874785 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: X-Tenant-Id: f1df9f889b3642a7ac056884bc2c51a4 Aug 27 02:16:01.874785 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: X-Tenant-Name: admin Aug 27 02:16:01.874785 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: X-User: admin Aug 27 02:16:01.874785 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: X-User-Domain-Id: default Aug 27 02:16:01.874785 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: X-User-Domain-Name: Default Aug 27 02:16:01.874785 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: X-User-Id: 6648709a4b6d470c80e9b42cb0d345ad Aug 27 02:16:01.874785 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: X-User-Name: admin Aug 27 02:16:01.874785 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: method: > Aug 27 02:16:01.874785 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: exception: Flavor 42 could not be found. Aug 27 02:16:01.874785 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: action_args: {'id': '42'} Aug 27 02:16:01.874785 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: {{(pid=105770) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Aug 27 02:16:01.874785 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 27 02:16:01.875732 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 66, in show Aug 27 02:16:01.875732 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: ERROR nova.api.openstack.wsgi flavor = flavors.get_flavor_by_flavor_id(id, ctxt=context) Aug 27 02:16:01.875732 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/flavors.py", line 139, in get_flavor_by_flavor_id Aug 27 02:16:01.875732 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: ERROR nova.api.openstack.wsgi return objects.Flavor.get_by_flavor_id(ctxt, flavorid, read_deleted) Aug 27 02:16:01.875732 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_versionedobjects/base.py", line 184, in wrapper Aug 27 02:16:01.875732 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: ERROR nova.api.openstack.wsgi result = fn(cls, context, *args, **kwargs) Aug 27 02:16:01.875732 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 390, in get_by_flavor_id Aug 27 02:16:01.875732 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: ERROR nova.api.openstack.wsgi db_flavor = cls._flavor_get_by_flavor_id_from_db(context, Aug 27 02:16:01.875732 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/db/utils.py", line 35, in wrapper Aug 27 02:16:01.875732 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Aug 27 02:16:01.875732 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 317, in _flavor_get_by_flavor_id_from_db Aug 27 02:16:01.875732 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: ERROR nova.api.openstack.wsgi raise exception.FlavorNotFound(flavor_id=flavor_id) Aug 27 02:16:01.875732 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: ERROR nova.api.openstack.wsgi nova.exception.FlavorNotFound: Flavor 42 could not be found. Aug 27 02:16:01.875732 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: ERROR nova.api.openstack.wsgi Aug 27 02:16:01.875732 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 27 02:16:01.875732 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: ERROR nova.api.openstack.wsgi Aug 27 02:16:01.875732 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 27 02:16:01.875732 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 27 02:16:01.875732 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 27 02:16:01.877002 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Aug 27 02:16:01.877002 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Aug 27 02:16:01.877002 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Aug 27 02:16:01.877002 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Aug 27 02:16:01.877002 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 68, in show Aug 27 02:16:01.877002 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPNotFound(explanation=e.format_message()) Aug 27 02:16:01.877002 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Flavor 42 could not be found. Aug 27 02:16:01.877002 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: ERROR nova.api.openstack.wsgi Aug 27 02:16:01.877002 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 27 02:16:01.877002 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: ERROR nova.api.openstack.wsgi Aug 27 02:16:01.877002 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 27 02:16:01.877002 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 27 02:16:01.877002 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 27 02:16:01.877002 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Aug 27 02:16:01.877002 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Aug 27 02:16:01.877002 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Flavor 42 could not be found. Aug 27 02:16:01.877002 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: ERROR nova.api.openstack.wsgi Aug 27 02:16:01.877002 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi [None req-e18e551f-0027-461a-bd65-24d4f09fdaaa admin admin] Returning 404 to user: Flavor 42 could not be found. {{(pid=105770) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Aug 27 02:16:01.878342 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: INFO nova.api.openstack.requestlog [None req-e18e551f-0027-461a-bd65-24d4f09fdaaa admin admin] 10.222.0.52 "GET /compute/v2.1/flavors/42" status: 404 len: 75 microversion: 2.90 time: 0.015866 Aug 27 02:16:01.878342 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: [pid: 105770|app: 0|req: 21/42] 10.222.0.52 () {60 vars in 1190 bytes} [Sat Aug 27 02:16:01 2022] GET /compute/v2.1/flavors/42 => generated 75 bytes in 17 msecs (HTTP/1.1 404) 9 headers in 380 bytes (1 switches on core 0) Aug 27 02:16:02.879495 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi [None req-cdebfe6c-729c-4762-9578-a75236ee093a admin admin] Calling method '>' {{(pid=105769) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:16:02.880491 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: INFO nova.api.openstack.requestlog [None req-cdebfe6c-729c-4762-9578-a75236ee093a admin admin] 10.222.0.52 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001454 Aug 27 02:16:02.880858 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: [pid: 105769|app: 0|req: 22/43] 10.222.0.52 () {56 vars in 1074 bytes} [Sat Aug 27 02:16:02 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 27 02:16:03.477576 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi [None req-a9a84cbd-a624-4037-8923-2e7e09b2cdb0 admin admin] Calling method '>' {{(pid=105770) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:16:03.484406 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: INFO nova.api.openstack.wsgi [None req-a9a84cbd-a624-4037-8923-2e7e09b2cdb0 admin admin] HTTP exception thrown: Flavor 84 could not be found. Aug 27 02:16:03.485955 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi [None req-a9a84cbd-a624-4037-8923-2e7e09b2cdb0 admin admin] Request method failure captured: Aug 27 02:16:03.485955 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: request: GET /compute/v2.1/flavors/84 HTTP/1.1 Aug 27 02:16:03.485955 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: Accept: application/json Aug 27 02:16:03.485955 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: Accept-Encoding: gzip, deflate Aug 27 02:16:03.485955 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: Connection: keep-alive Aug 27 02:16:03.485955 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: Content-Length: 0 Aug 27 02:16:03.485955 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: Host: 10.222.0.52 Aug 27 02:16:03.485955 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: Openstack-Api-Version: compute 2.90 Aug 27 02:16:03.485955 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: Openstack-System-Scope: None Aug 27 02:16:03.485955 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: User-Agent: python-novaclient Aug 27 02:16:03.485955 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: X-Auth-Token: gAAAAABjCX5inEYF9bCCImaWzL8XMJYTv3IeFmcUbi6Q9CiUS9wekeIo8LfXSJtfv7seESaV-2LRRU-0J5qBr5s3f53XlijPIoQtuM8BfvK_mAw2CJ4E0UqNFc-Dzj3mRFiCNcorPMS95RVw5A4KnuMIuPq7FPk1imAHgIxbUsXBlK5GGCIpiFo Aug 27 02:16:03.485955 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: X-Domain-Id: None Aug 27 02:16:03.485955 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: X-Domain-Name: None Aug 27 02:16:03.485955 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: X-Identity-Status: Confirmed Aug 27 02:16:03.485955 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: X-Is-Admin-Project: True Aug 27 02:16:03.485955 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: X-Openstack-Nova-Api-Version: 2.90 Aug 27 02:16:03.485955 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: X-Project-Domain-Id: default Aug 27 02:16:03.485955 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: X-Project-Domain-Name: Default Aug 27 02:16:03.485955 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: X-Project-Id: f1df9f889b3642a7ac056884bc2c51a4 Aug 27 02:16:03.485955 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: X-Project-Name: admin Aug 27 02:16:03.485955 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: X-Role: reader,admin,member Aug 27 02:16:03.485955 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: X-Roles: reader,admin,member Aug 27 02:16:03.487490 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: X-Service-Catalog: [{"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.52:8080", "publicURL": "http://10.222.0.52:8080/v1/AUTH_f1df9f889b3642a7ac056884bc2c51a4"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52:9696/networking"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/compute/v2/f1df9f889b3642a7ac056884bc2c51a4"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/image"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/volume/v3/f1df9f889b3642a7ac056884bc2c51a4"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/identity"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/volume/v3/f1df9f889b3642a7ac056884bc2c51a4"}]}] Aug 27 02:16:03.487490 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: X-Tenant: admin Aug 27 02:16:03.487490 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: X-Tenant-Id: f1df9f889b3642a7ac056884bc2c51a4 Aug 27 02:16:03.487490 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: X-Tenant-Name: admin Aug 27 02:16:03.487490 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: X-User: admin Aug 27 02:16:03.487490 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: X-User-Domain-Id: default Aug 27 02:16:03.487490 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: X-User-Domain-Name: Default Aug 27 02:16:03.487490 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: X-User-Id: 6648709a4b6d470c80e9b42cb0d345ad Aug 27 02:16:03.487490 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: X-User-Name: admin Aug 27 02:16:03.487490 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: method: > Aug 27 02:16:03.487490 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: exception: Flavor 84 could not be found. Aug 27 02:16:03.487490 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: action_args: {'id': '84'} Aug 27 02:16:03.487490 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: {{(pid=105770) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Aug 27 02:16:03.487490 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 27 02:16:03.488378 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 66, in show Aug 27 02:16:03.488378 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: ERROR nova.api.openstack.wsgi flavor = flavors.get_flavor_by_flavor_id(id, ctxt=context) Aug 27 02:16:03.488378 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/flavors.py", line 139, in get_flavor_by_flavor_id Aug 27 02:16:03.488378 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: ERROR nova.api.openstack.wsgi return objects.Flavor.get_by_flavor_id(ctxt, flavorid, read_deleted) Aug 27 02:16:03.488378 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_versionedobjects/base.py", line 184, in wrapper Aug 27 02:16:03.488378 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: ERROR nova.api.openstack.wsgi result = fn(cls, context, *args, **kwargs) Aug 27 02:16:03.488378 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 390, in get_by_flavor_id Aug 27 02:16:03.488378 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: ERROR nova.api.openstack.wsgi db_flavor = cls._flavor_get_by_flavor_id_from_db(context, Aug 27 02:16:03.488378 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/db/utils.py", line 35, in wrapper Aug 27 02:16:03.488378 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Aug 27 02:16:03.488378 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 317, in _flavor_get_by_flavor_id_from_db Aug 27 02:16:03.488378 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: ERROR nova.api.openstack.wsgi raise exception.FlavorNotFound(flavor_id=flavor_id) Aug 27 02:16:03.488378 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: ERROR nova.api.openstack.wsgi nova.exception.FlavorNotFound: Flavor 84 could not be found. Aug 27 02:16:03.488378 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: ERROR nova.api.openstack.wsgi Aug 27 02:16:03.488378 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 27 02:16:03.488378 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: ERROR nova.api.openstack.wsgi Aug 27 02:16:03.488378 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 27 02:16:03.488378 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 27 02:16:03.488378 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 27 02:16:03.489862 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Aug 27 02:16:03.489862 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Aug 27 02:16:03.489862 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Aug 27 02:16:03.489862 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Aug 27 02:16:03.489862 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 68, in show Aug 27 02:16:03.489862 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPNotFound(explanation=e.format_message()) Aug 27 02:16:03.489862 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Flavor 84 could not be found. Aug 27 02:16:03.489862 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: ERROR nova.api.openstack.wsgi Aug 27 02:16:03.489862 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 27 02:16:03.489862 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: ERROR nova.api.openstack.wsgi Aug 27 02:16:03.489862 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 27 02:16:03.489862 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 27 02:16:03.489862 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 27 02:16:03.489862 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Aug 27 02:16:03.489862 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Aug 27 02:16:03.489862 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Flavor 84 could not be found. Aug 27 02:16:03.489862 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: ERROR nova.api.openstack.wsgi Aug 27 02:16:03.489862 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi [None req-a9a84cbd-a624-4037-8923-2e7e09b2cdb0 admin admin] Returning 404 to user: Flavor 84 could not be found. {{(pid=105770) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Aug 27 02:16:03.490872 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: INFO nova.api.openstack.requestlog [None req-a9a84cbd-a624-4037-8923-2e7e09b2cdb0 admin admin] 10.222.0.52 "GET /compute/v2.1/flavors/84" status: 404 len: 75 microversion: 2.90 time: 0.464910 Aug 27 02:16:03.490872 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: [pid: 105770|app: 0|req: 22/44] 10.222.0.52 () {60 vars in 1190 bytes} [Sat Aug 27 02:16:03 2022] GET /compute/v2.1/flavors/84 => generated 75 bytes in 466 msecs (HTTP/1.1 404) 9 headers in 380 bytes (1 switches on core 0) Aug 27 02:16:03.497697 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi [None req-772fe98f-e58a-4d27-b6f0-24023ffe37c7 admin admin] Calling method '>' {{(pid=105769) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:16:03.513832 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: INFO nova.api.openstack.requestlog [None req-772fe98f-e58a-4d27-b6f0-24023ffe37c7 admin admin] 10.222.0.52 "GET /compute/v2.1/flavors?is_public=None" status: 200 len: 2056 microversion: 2.90 time: 0.019686 Aug 27 02:16:03.514394 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: [pid: 105769|app: 0|req: 23/45] 10.222.0.52 () {60 vars in 1210 bytes} [Sat Aug 27 02:16:03 2022] GET /compute/v2.1/flavors?is_public=None => generated 2056 bytes in 21 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 27 02:16:03.524437 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi [None req-472bc602-6067-4868-a47e-37f96c6bbddb admin admin] Calling method '>' {{(pid=105770) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:16:03.541884 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: INFO nova.api.openstack.requestlog [None req-472bc602-6067-4868-a47e-37f96c6bbddb admin admin] 10.222.0.52 "GET /compute/v2.1/flavors?is_public=None" status: 200 len: 2056 microversion: 2.90 time: 0.020911 Aug 27 02:16:03.542449 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: [pid: 105770|app: 0|req: 23/46] 10.222.0.52 () {60 vars in 1210 bytes} [Sat Aug 27 02:16:03 2022] GET /compute/v2.1/flavors?is_public=None => generated 2056 bytes in 22 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 27 02:16:03.552879 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi [None req-5df00163-bcaf-4493-b49e-74c6a8e12156 admin admin] Calling method '>' {{(pid=105769) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:16:03.559242 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: INFO nova.api.openstack.wsgi [None req-5df00163-bcaf-4493-b49e-74c6a8e12156 admin admin] HTTP exception thrown: Flavor 84 could not be found. Aug 27 02:16:03.560764 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi [None req-5df00163-bcaf-4493-b49e-74c6a8e12156 admin admin] Request method failure captured: Aug 27 02:16:03.560764 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: request: GET /compute/v2.1/flavors/84 HTTP/1.1 Aug 27 02:16:03.560764 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: Accept: application/json Aug 27 02:16:03.560764 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: Accept-Encoding: gzip, deflate Aug 27 02:16:03.560764 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: Connection: keep-alive Aug 27 02:16:03.560764 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: Content-Length: 0 Aug 27 02:16:03.560764 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: Host: 10.222.0.52 Aug 27 02:16:03.560764 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: Openstack-Api-Version: compute 2.90 Aug 27 02:16:03.560764 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: Openstack-System-Scope: None Aug 27 02:16:03.560764 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: User-Agent: python-novaclient Aug 27 02:16:03.560764 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: X-Auth-Token: gAAAAABjCX5inEYF9bCCImaWzL8XMJYTv3IeFmcUbi6Q9CiUS9wekeIo8LfXSJtfv7seESaV-2LRRU-0J5qBr5s3f53XlijPIoQtuM8BfvK_mAw2CJ4E0UqNFc-Dzj3mRFiCNcorPMS95RVw5A4KnuMIuPq7FPk1imAHgIxbUsXBlK5GGCIpiFo Aug 27 02:16:03.560764 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: X-Domain-Id: None Aug 27 02:16:03.560764 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: X-Domain-Name: None Aug 27 02:16:03.560764 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: X-Identity-Status: Confirmed Aug 27 02:16:03.560764 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: X-Is-Admin-Project: True Aug 27 02:16:03.560764 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: X-Openstack-Nova-Api-Version: 2.90 Aug 27 02:16:03.560764 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: X-Project-Domain-Id: default Aug 27 02:16:03.560764 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: X-Project-Domain-Name: Default Aug 27 02:16:03.560764 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: X-Project-Id: f1df9f889b3642a7ac056884bc2c51a4 Aug 27 02:16:03.560764 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: X-Project-Name: admin Aug 27 02:16:03.560764 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: X-Role: reader,admin,member Aug 27 02:16:03.560764 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: X-Roles: reader,admin,member Aug 27 02:16:03.562377 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: X-Service-Catalog: [{"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.52:8080", "publicURL": "http://10.222.0.52:8080/v1/AUTH_f1df9f889b3642a7ac056884bc2c51a4"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52:9696/networking"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/compute/v2/f1df9f889b3642a7ac056884bc2c51a4"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/image"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/volume/v3/f1df9f889b3642a7ac056884bc2c51a4"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/identity"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/volume/v3/f1df9f889b3642a7ac056884bc2c51a4"}]}] Aug 27 02:16:03.562377 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: X-Tenant: admin Aug 27 02:16:03.562377 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: X-Tenant-Id: f1df9f889b3642a7ac056884bc2c51a4 Aug 27 02:16:03.562377 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: X-Tenant-Name: admin Aug 27 02:16:03.562377 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: X-User: admin Aug 27 02:16:03.562377 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: X-User-Domain-Id: default Aug 27 02:16:03.562377 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: X-User-Domain-Name: Default Aug 27 02:16:03.562377 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: X-User-Id: 6648709a4b6d470c80e9b42cb0d345ad Aug 27 02:16:03.562377 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: X-User-Name: admin Aug 27 02:16:03.562377 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: method: > Aug 27 02:16:03.562377 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: exception: Flavor 84 could not be found. Aug 27 02:16:03.562377 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: action_args: {'id': '84'} Aug 27 02:16:03.562377 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: {{(pid=105769) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Aug 27 02:16:03.562377 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 27 02:16:03.563234 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 66, in show Aug 27 02:16:03.563234 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: ERROR nova.api.openstack.wsgi flavor = flavors.get_flavor_by_flavor_id(id, ctxt=context) Aug 27 02:16:03.563234 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/flavors.py", line 139, in get_flavor_by_flavor_id Aug 27 02:16:03.563234 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: ERROR nova.api.openstack.wsgi return objects.Flavor.get_by_flavor_id(ctxt, flavorid, read_deleted) Aug 27 02:16:03.563234 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_versionedobjects/base.py", line 184, in wrapper Aug 27 02:16:03.563234 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: ERROR nova.api.openstack.wsgi result = fn(cls, context, *args, **kwargs) Aug 27 02:16:03.563234 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 390, in get_by_flavor_id Aug 27 02:16:03.563234 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: ERROR nova.api.openstack.wsgi db_flavor = cls._flavor_get_by_flavor_id_from_db(context, Aug 27 02:16:03.563234 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/db/utils.py", line 35, in wrapper Aug 27 02:16:03.563234 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Aug 27 02:16:03.563234 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 317, in _flavor_get_by_flavor_id_from_db Aug 27 02:16:03.563234 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: ERROR nova.api.openstack.wsgi raise exception.FlavorNotFound(flavor_id=flavor_id) Aug 27 02:16:03.563234 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: ERROR nova.api.openstack.wsgi nova.exception.FlavorNotFound: Flavor 84 could not be found. Aug 27 02:16:03.563234 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: ERROR nova.api.openstack.wsgi Aug 27 02:16:03.563234 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 27 02:16:03.563234 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: ERROR nova.api.openstack.wsgi Aug 27 02:16:03.563234 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 27 02:16:03.563234 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 27 02:16:03.563234 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 27 02:16:03.564371 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Aug 27 02:16:03.564371 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Aug 27 02:16:03.564371 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Aug 27 02:16:03.564371 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Aug 27 02:16:03.564371 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 68, in show Aug 27 02:16:03.564371 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPNotFound(explanation=e.format_message()) Aug 27 02:16:03.564371 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Flavor 84 could not be found. Aug 27 02:16:03.564371 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: ERROR nova.api.openstack.wsgi Aug 27 02:16:03.564371 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 27 02:16:03.564371 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: ERROR nova.api.openstack.wsgi Aug 27 02:16:03.564371 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 27 02:16:03.564371 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 27 02:16:03.564371 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 27 02:16:03.564371 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Aug 27 02:16:03.564371 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Aug 27 02:16:03.564371 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Flavor 84 could not be found. Aug 27 02:16:03.564371 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: ERROR nova.api.openstack.wsgi Aug 27 02:16:03.564371 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi [None req-5df00163-bcaf-4493-b49e-74c6a8e12156 admin admin] Returning 404 to user: Flavor 84 could not be found. {{(pid=105769) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Aug 27 02:16:03.565616 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: INFO nova.api.openstack.requestlog [None req-5df00163-bcaf-4493-b49e-74c6a8e12156 admin admin] 10.222.0.52 "GET /compute/v2.1/flavors/84" status: 404 len: 75 microversion: 2.90 time: 0.012955 Aug 27 02:16:03.565616 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: [pid: 105769|app: 0|req: 24/47] 10.222.0.52 () {60 vars in 1190 bytes} [Sat Aug 27 02:16:03 2022] GET /compute/v2.1/flavors/84 => generated 75 bytes in 14 msecs (HTTP/1.1 404) 9 headers in 380 bytes (1 switches on core 0) Aug 27 02:16:04.600519 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi [None req-472bc602-6067-4868-a47e-37f96c6bbddb admin admin] Calling method '>' {{(pid=105770) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:16:04.601598 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: INFO nova.api.openstack.requestlog [None req-472bc602-6067-4868-a47e-37f96c6bbddb admin admin] 10.222.0.52 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001500 Aug 27 02:16:04.601949 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: [pid: 105770|app: 0|req: 24/48] 10.222.0.52 () {56 vars in 1074 bytes} [Sat Aug 27 02:16:04 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 27 02:16:05.194067 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi [None req-fa6582ae-2596-465a-8ea2-314af70f42d1 admin admin] Calling method '>' {{(pid=105769) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:16:05.201789 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: INFO nova.api.openstack.wsgi [None req-fa6582ae-2596-465a-8ea2-314af70f42d1 admin admin] HTTP exception thrown: Flavor 451 could not be found. Aug 27 02:16:05.203308 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi [None req-fa6582ae-2596-465a-8ea2-314af70f42d1 admin admin] Request method failure captured: Aug 27 02:16:05.203308 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: request: GET /compute/v2.1/flavors/451 HTTP/1.1 Aug 27 02:16:05.203308 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: Accept: application/json Aug 27 02:16:05.203308 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: Accept-Encoding: gzip, deflate Aug 27 02:16:05.203308 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: Connection: keep-alive Aug 27 02:16:05.203308 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: Content-Length: 0 Aug 27 02:16:05.203308 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: Host: 10.222.0.52 Aug 27 02:16:05.203308 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: Openstack-Api-Version: compute 2.90 Aug 27 02:16:05.203308 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: Openstack-System-Scope: None Aug 27 02:16:05.203308 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: User-Agent: python-novaclient Aug 27 02:16:05.203308 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: X-Auth-Token: gAAAAABjCX5k3Eq-Cyr2CKIM_1dgVDE4bq8NGi18_NtkdwAxdIJS1lSuyxXXubAZ8CNdp9vMEH2k6ptFrJDaAx0XatVLVq7kXpsbDK1oe5P6q_NHLd4LicSRQSs6NEiyEt9HoJ12ux7p4wbFbfru3EfaZWhle6huqzKvxh1J5wi1RfpI4gZso48 Aug 27 02:16:05.203308 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: X-Domain-Id: None Aug 27 02:16:05.203308 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: X-Domain-Name: None Aug 27 02:16:05.203308 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: X-Identity-Status: Confirmed Aug 27 02:16:05.203308 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: X-Is-Admin-Project: True Aug 27 02:16:05.203308 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: X-Openstack-Nova-Api-Version: 2.90 Aug 27 02:16:05.203308 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: X-Project-Domain-Id: default Aug 27 02:16:05.203308 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: X-Project-Domain-Name: Default Aug 27 02:16:05.203308 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: X-Project-Id: f1df9f889b3642a7ac056884bc2c51a4 Aug 27 02:16:05.203308 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: X-Project-Name: admin Aug 27 02:16:05.203308 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: X-Role: reader,admin,member Aug 27 02:16:05.203308 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: X-Roles: reader,admin,member Aug 27 02:16:05.204782 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: X-Service-Catalog: [{"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.52:8080", "publicURL": "http://10.222.0.52:8080/v1/AUTH_f1df9f889b3642a7ac056884bc2c51a4"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52:9696/networking"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/compute/v2/f1df9f889b3642a7ac056884bc2c51a4"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/image"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/volume/v3/f1df9f889b3642a7ac056884bc2c51a4"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/identity"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/volume/v3/f1df9f889b3642a7ac056884bc2c51a4"}]}] Aug 27 02:16:05.204782 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: X-Tenant: admin Aug 27 02:16:05.204782 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: X-Tenant-Id: f1df9f889b3642a7ac056884bc2c51a4 Aug 27 02:16:05.204782 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: X-Tenant-Name: admin Aug 27 02:16:05.204782 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: X-User: admin Aug 27 02:16:05.204782 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: X-User-Domain-Id: default Aug 27 02:16:05.204782 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: X-User-Domain-Name: Default Aug 27 02:16:05.204782 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: X-User-Id: 6648709a4b6d470c80e9b42cb0d345ad Aug 27 02:16:05.204782 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: X-User-Name: admin Aug 27 02:16:05.204782 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: method: > Aug 27 02:16:05.204782 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: exception: Flavor 451 could not be found. Aug 27 02:16:05.204782 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: action_args: {'id': '451'} Aug 27 02:16:05.204782 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: {{(pid=105769) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Aug 27 02:16:05.204782 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 27 02:16:05.205598 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 66, in show Aug 27 02:16:05.205598 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: ERROR nova.api.openstack.wsgi flavor = flavors.get_flavor_by_flavor_id(id, ctxt=context) Aug 27 02:16:05.205598 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/flavors.py", line 139, in get_flavor_by_flavor_id Aug 27 02:16:05.205598 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: ERROR nova.api.openstack.wsgi return objects.Flavor.get_by_flavor_id(ctxt, flavorid, read_deleted) Aug 27 02:16:05.205598 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_versionedobjects/base.py", line 184, in wrapper Aug 27 02:16:05.205598 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: ERROR nova.api.openstack.wsgi result = fn(cls, context, *args, **kwargs) Aug 27 02:16:05.205598 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 390, in get_by_flavor_id Aug 27 02:16:05.205598 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: ERROR nova.api.openstack.wsgi db_flavor = cls._flavor_get_by_flavor_id_from_db(context, Aug 27 02:16:05.205598 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/db/utils.py", line 35, in wrapper Aug 27 02:16:05.205598 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Aug 27 02:16:05.205598 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 317, in _flavor_get_by_flavor_id_from_db Aug 27 02:16:05.205598 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: ERROR nova.api.openstack.wsgi raise exception.FlavorNotFound(flavor_id=flavor_id) Aug 27 02:16:05.205598 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: ERROR nova.api.openstack.wsgi nova.exception.FlavorNotFound: Flavor 451 could not be found. Aug 27 02:16:05.205598 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: ERROR nova.api.openstack.wsgi Aug 27 02:16:05.205598 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 27 02:16:05.205598 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: ERROR nova.api.openstack.wsgi Aug 27 02:16:05.205598 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 27 02:16:05.205598 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 27 02:16:05.205598 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 27 02:16:05.206686 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Aug 27 02:16:05.206686 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Aug 27 02:16:05.206686 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Aug 27 02:16:05.206686 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Aug 27 02:16:05.206686 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 68, in show Aug 27 02:16:05.206686 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPNotFound(explanation=e.format_message()) Aug 27 02:16:05.206686 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Flavor 451 could not be found. Aug 27 02:16:05.206686 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: ERROR nova.api.openstack.wsgi Aug 27 02:16:05.206686 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 27 02:16:05.206686 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: ERROR nova.api.openstack.wsgi Aug 27 02:16:05.206686 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 27 02:16:05.206686 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 27 02:16:05.206686 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 27 02:16:05.206686 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Aug 27 02:16:05.206686 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Aug 27 02:16:05.206686 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Flavor 451 could not be found. Aug 27 02:16:05.206686 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: ERROR nova.api.openstack.wsgi Aug 27 02:16:05.206686 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi [None req-fa6582ae-2596-465a-8ea2-314af70f42d1 admin admin] Returning 404 to user: Flavor 451 could not be found. {{(pid=105769) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Aug 27 02:16:05.207752 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: INFO nova.api.openstack.requestlog [None req-fa6582ae-2596-465a-8ea2-314af70f42d1 admin admin] 10.222.0.52 "GET /compute/v2.1/flavors/451" status: 404 len: 76 microversion: 2.90 time: 0.457816 Aug 27 02:16:05.207752 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: [pid: 105769|app: 0|req: 25/49] 10.222.0.52 () {60 vars in 1193 bytes} [Sat Aug 27 02:16:04 2022] GET /compute/v2.1/flavors/451 => generated 76 bytes in 459 msecs (HTTP/1.1 404) 9 headers in 380 bytes (1 switches on core 0) Aug 27 02:16:05.216226 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi [None req-1e14bcae-1af3-4a8f-869b-78cebb638472 admin admin] Calling method '>' {{(pid=105770) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:16:05.232263 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: INFO nova.api.openstack.requestlog [None req-1e14bcae-1af3-4a8f-869b-78cebb638472 admin admin] 10.222.0.52 "GET /compute/v2.1/flavors?is_public=None" status: 200 len: 2056 microversion: 2.90 time: 0.020187 Aug 27 02:16:05.232806 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: [pid: 105770|app: 0|req: 25/50] 10.222.0.52 () {60 vars in 1210 bytes} [Sat Aug 27 02:16:05 2022] GET /compute/v2.1/flavors?is_public=None => generated 2056 bytes in 21 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 27 02:16:05.242997 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi [None req-92c73149-2af0-44e5-bc98-9a8a1fe34dc4 admin admin] Calling method '>' {{(pid=105769) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:16:05.259967 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: INFO nova.api.openstack.requestlog [None req-92c73149-2af0-44e5-bc98-9a8a1fe34dc4 admin admin] 10.222.0.52 "GET /compute/v2.1/flavors?is_public=None" status: 200 len: 2056 microversion: 2.90 time: 0.020423 Aug 27 02:16:05.260473 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: [pid: 105769|app: 0|req: 26/51] 10.222.0.52 () {60 vars in 1210 bytes} [Sat Aug 27 02:16:05 2022] GET /compute/v2.1/flavors?is_public=None => generated 2056 bytes in 22 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 27 02:16:05.270771 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi [None req-43aff990-cf31-4dd0-8624-ff01513a7543 admin admin] Calling method '>' {{(pid=105770) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:16:05.276491 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: INFO nova.api.openstack.wsgi [None req-43aff990-cf31-4dd0-8624-ff01513a7543 admin admin] HTTP exception thrown: Flavor 451 could not be found. Aug 27 02:16:05.278724 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi [None req-43aff990-cf31-4dd0-8624-ff01513a7543 admin admin] Request method failure captured: Aug 27 02:16:05.278724 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: request: GET /compute/v2.1/flavors/451 HTTP/1.1 Aug 27 02:16:05.278724 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: Accept: application/json Aug 27 02:16:05.278724 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: Accept-Encoding: gzip, deflate Aug 27 02:16:05.278724 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: Connection: keep-alive Aug 27 02:16:05.278724 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: Content-Length: 0 Aug 27 02:16:05.278724 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: Host: 10.222.0.52 Aug 27 02:16:05.278724 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: Openstack-Api-Version: compute 2.90 Aug 27 02:16:05.278724 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: Openstack-System-Scope: None Aug 27 02:16:05.278724 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: User-Agent: python-novaclient Aug 27 02:16:05.278724 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: X-Auth-Token: gAAAAABjCX5k3Eq-Cyr2CKIM_1dgVDE4bq8NGi18_NtkdwAxdIJS1lSuyxXXubAZ8CNdp9vMEH2k6ptFrJDaAx0XatVLVq7kXpsbDK1oe5P6q_NHLd4LicSRQSs6NEiyEt9HoJ12ux7p4wbFbfru3EfaZWhle6huqzKvxh1J5wi1RfpI4gZso48 Aug 27 02:16:05.278724 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: X-Domain-Id: None Aug 27 02:16:05.278724 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: X-Domain-Name: None Aug 27 02:16:05.278724 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: X-Identity-Status: Confirmed Aug 27 02:16:05.278724 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: X-Is-Admin-Project: True Aug 27 02:16:05.278724 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: X-Openstack-Nova-Api-Version: 2.90 Aug 27 02:16:05.278724 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: X-Project-Domain-Id: default Aug 27 02:16:05.278724 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: X-Project-Domain-Name: Default Aug 27 02:16:05.278724 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: X-Project-Id: f1df9f889b3642a7ac056884bc2c51a4 Aug 27 02:16:05.278724 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: X-Project-Name: admin Aug 27 02:16:05.278724 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: X-Role: reader,admin,member Aug 27 02:16:05.278724 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: X-Roles: reader,admin,member Aug 27 02:16:05.280223 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: X-Service-Catalog: [{"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.52:8080", "publicURL": "http://10.222.0.52:8080/v1/AUTH_f1df9f889b3642a7ac056884bc2c51a4"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52:9696/networking"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/compute/v2/f1df9f889b3642a7ac056884bc2c51a4"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/image"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/volume/v3/f1df9f889b3642a7ac056884bc2c51a4"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/identity"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/volume/v3/f1df9f889b3642a7ac056884bc2c51a4"}]}] Aug 27 02:16:05.280223 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: X-Tenant: admin Aug 27 02:16:05.280223 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: X-Tenant-Id: f1df9f889b3642a7ac056884bc2c51a4 Aug 27 02:16:05.280223 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: X-Tenant-Name: admin Aug 27 02:16:05.280223 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: X-User: admin Aug 27 02:16:05.280223 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: X-User-Domain-Id: default Aug 27 02:16:05.280223 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: X-User-Domain-Name: Default Aug 27 02:16:05.280223 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: X-User-Id: 6648709a4b6d470c80e9b42cb0d345ad Aug 27 02:16:05.280223 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: X-User-Name: admin Aug 27 02:16:05.280223 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: method: > Aug 27 02:16:05.280223 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: exception: Flavor 451 could not be found. Aug 27 02:16:05.280223 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: action_args: {'id': '451'} Aug 27 02:16:05.280223 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: {{(pid=105770) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Aug 27 02:16:05.280223 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 27 02:16:05.281021 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 66, in show Aug 27 02:16:05.281021 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: ERROR nova.api.openstack.wsgi flavor = flavors.get_flavor_by_flavor_id(id, ctxt=context) Aug 27 02:16:05.281021 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/flavors.py", line 139, in get_flavor_by_flavor_id Aug 27 02:16:05.281021 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: ERROR nova.api.openstack.wsgi return objects.Flavor.get_by_flavor_id(ctxt, flavorid, read_deleted) Aug 27 02:16:05.281021 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_versionedobjects/base.py", line 184, in wrapper Aug 27 02:16:05.281021 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: ERROR nova.api.openstack.wsgi result = fn(cls, context, *args, **kwargs) Aug 27 02:16:05.281021 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 390, in get_by_flavor_id Aug 27 02:16:05.281021 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: ERROR nova.api.openstack.wsgi db_flavor = cls._flavor_get_by_flavor_id_from_db(context, Aug 27 02:16:05.281021 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/db/utils.py", line 35, in wrapper Aug 27 02:16:05.281021 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Aug 27 02:16:05.281021 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 317, in _flavor_get_by_flavor_id_from_db Aug 27 02:16:05.281021 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: ERROR nova.api.openstack.wsgi raise exception.FlavorNotFound(flavor_id=flavor_id) Aug 27 02:16:05.281021 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: ERROR nova.api.openstack.wsgi nova.exception.FlavorNotFound: Flavor 451 could not be found. Aug 27 02:16:05.281021 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: ERROR nova.api.openstack.wsgi Aug 27 02:16:05.281021 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 27 02:16:05.281021 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: ERROR nova.api.openstack.wsgi Aug 27 02:16:05.281021 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 27 02:16:05.281021 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 27 02:16:05.281021 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 27 02:16:05.282343 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Aug 27 02:16:05.282343 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Aug 27 02:16:05.282343 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Aug 27 02:16:05.282343 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Aug 27 02:16:05.282343 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 68, in show Aug 27 02:16:05.282343 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPNotFound(explanation=e.format_message()) Aug 27 02:16:05.282343 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Flavor 451 could not be found. Aug 27 02:16:05.282343 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: ERROR nova.api.openstack.wsgi Aug 27 02:16:05.282343 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 27 02:16:05.282343 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: ERROR nova.api.openstack.wsgi Aug 27 02:16:05.282343 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 27 02:16:05.282343 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 27 02:16:05.282343 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 27 02:16:05.282343 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Aug 27 02:16:05.282343 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Aug 27 02:16:05.282343 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Flavor 451 could not be found. Aug 27 02:16:05.282343 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: ERROR nova.api.openstack.wsgi Aug 27 02:16:05.282343 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi [None req-43aff990-cf31-4dd0-8624-ff01513a7543 admin admin] Returning 404 to user: Flavor 451 could not be found. {{(pid=105770) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Aug 27 02:16:05.283554 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: INFO nova.api.openstack.requestlog [None req-43aff990-cf31-4dd0-8624-ff01513a7543 admin admin] 10.222.0.52 "GET /compute/v2.1/flavors/451" status: 404 len: 76 microversion: 2.90 time: 0.013013 Aug 27 02:16:05.283554 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: [pid: 105770|app: 0|req: 26/52] 10.222.0.52 () {60 vars in 1193 bytes} [Sat Aug 27 02:16:05 2022] GET /compute/v2.1/flavors/451 => generated 76 bytes in 14 msecs (HTTP/1.1 404) 9 headers in 380 bytes (2 switches on core 0) Aug 27 02:16:06.287140 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi [None req-92c73149-2af0-44e5-bc98-9a8a1fe34dc4 admin admin] Calling method '>' {{(pid=105769) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:16:06.288137 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: INFO nova.api.openstack.requestlog [None req-92c73149-2af0-44e5-bc98-9a8a1fe34dc4 admin admin] 10.222.0.52 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001532 Aug 27 02:16:06.288523 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: [pid: 105769|app: 0|req: 27/53] 10.222.0.52 () {56 vars in 1074 bytes} [Sat Aug 27 02:16:06 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Aug 27 02:16:06.876826 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi [None req-a55d0100-91ee-45d2-9ed9-a02ea0808020 admin admin] Action: 'create', calling method: >, body: {"flavor": {"name": "m1.nano", "ram": 256, "vcpus": 1, "disk": 1, "id": "42", "swap": 0, "OS-FLV-EXT-DATA:ephemeral": 0, "rxtx_factor": 1.0, "os-flavor-access:is_public": true}} {{(pid=105770) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 27 02:16:06.898811 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: INFO nova.api.openstack.requestlog [None req-a55d0100-91ee-45d2-9ed9-a02ea0808020 admin admin] 10.222.0.52 "POST /compute/v2.1/flavors" status: 200 len: 404 microversion: 2.90 time: 0.465981 Aug 27 02:16:06.899398 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: [pid: 105770|app: 0|req: 27/54] 10.222.0.52 () {64 vars in 1235 bytes} [Sat Aug 27 02:16:06 2022] POST /compute/v2.1/flavors => generated 404 bytes in 467 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Aug 27 02:16:07.943810 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi [None req-92c73149-2af0-44e5-bc98-9a8a1fe34dc4 admin admin] Calling method '>' {{(pid=105769) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:16:07.944881 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: INFO nova.api.openstack.requestlog [None req-92c73149-2af0-44e5-bc98-9a8a1fe34dc4 admin admin] 10.222.0.52 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001467 Aug 27 02:16:07.945515 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: [pid: 105769|app: 0|req: 28/55] 10.222.0.52 () {56 vars in 1074 bytes} [Sat Aug 27 02:16:07 2022] GET /compute/v2.1 => generated 388 bytes in 3 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 27 02:16:08.552753 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi [None req-9e9584bc-f7d0-408a-853f-c433572c53bf admin admin] Action: 'create', calling method: >, body: {"flavor": {"name": "m1.micro", "ram": 256, "vcpus": 1, "disk": 2, "id": "84", "swap": 0, "OS-FLV-EXT-DATA:ephemeral": 0, "rxtx_factor": 1.0, "os-flavor-access:is_public": true}} {{(pid=105770) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 27 02:16:08.575652 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: INFO nova.api.openstack.requestlog [None req-9e9584bc-f7d0-408a-853f-c433572c53bf admin admin] 10.222.0.52 "POST /compute/v2.1/flavors" status: 200 len: 405 microversion: 2.90 time: 0.477644 Aug 27 02:16:08.576205 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: [pid: 105770|app: 0|req: 28/56] 10.222.0.52 () {64 vars in 1235 bytes} [Sat Aug 27 02:16:08 2022] POST /compute/v2.1/flavors => generated 405 bytes in 479 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Aug 27 02:16:09.581551 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi [None req-92c73149-2af0-44e5-bc98-9a8a1fe34dc4 admin admin] Calling method '>' {{(pid=105769) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:16:09.582548 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: INFO nova.api.openstack.requestlog [None req-92c73149-2af0-44e5-bc98-9a8a1fe34dc4 admin admin] 10.222.0.52 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001479 Aug 27 02:16:09.582944 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: [pid: 105769|app: 0|req: 29/57] 10.222.0.52 () {56 vars in 1074 bytes} [Sat Aug 27 02:16:09 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 27 02:16:10.159972 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi [None req-66974611-5704-436e-b7ad-3c18548f4350 admin admin] Action: 'create', calling method: >, body: {"flavor": {"name": "m1.heat", "ram": 512, "vcpus": 1, "disk": 1, "id": "451", "swap": 0, "OS-FLV-EXT-DATA:ephemeral": 0, "rxtx_factor": 1.0, "os-flavor-access:is_public": true}} {{(pid=105770) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 27 02:16:10.182854 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: INFO nova.api.openstack.requestlog [None req-66974611-5704-436e-b7ad-3c18548f4350 admin admin] 10.222.0.52 "POST /compute/v2.1/flavors" status: 200 len: 407 microversion: 2.90 time: 0.457367 Aug 27 02:16:10.183480 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: [pid: 105770|app: 0|req: 29/58] 10.222.0.52 () {64 vars in 1235 bytes} [Sat Aug 27 02:16:09 2022] POST /compute/v2.1/flavors => generated 407 bytes in 459 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Aug 27 02:16:11.246312 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi [None req-92c73149-2af0-44e5-bc98-9a8a1fe34dc4 admin admin] Calling method '>' {{(pid=105769) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:16:11.247814 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: INFO nova.api.openstack.requestlog [None req-92c73149-2af0-44e5-bc98-9a8a1fe34dc4 admin admin] 10.222.0.52 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.002031 Aug 27 02:16:11.248324 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: [pid: 105769|app: 0|req: 30/59] 10.222.0.52 () {56 vars in 1074 bytes} [Sat Aug 27 02:16:11 2022] GET /compute/v2.1 => generated 388 bytes in 3 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 27 02:16:11.823027 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi [None req-797634f1-3e32-4fa9-b6e1-9009be15d5f9 admin admin] Action: 'create', calling method: >, body: {"flavor": {"name": "m1.heat2", "ram": 1024, "vcpus": 1, "disk": 1, "id": "452", "swap": 0, "OS-FLV-EXT-DATA:ephemeral": 0, "rxtx_factor": 1.0, "os-flavor-access:is_public": true}} {{(pid=105770) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 27 02:16:11.846499 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: INFO nova.api.openstack.requestlog [None req-797634f1-3e32-4fa9-b6e1-9009be15d5f9 admin admin] 10.222.0.52 "POST /compute/v2.1/flavors" status: 200 len: 409 microversion: 2.90 time: 0.459626 Aug 27 02:16:11.847055 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: [pid: 105770|app: 0|req: 30/60] 10.222.0.52 () {64 vars in 1235 bytes} [Sat Aug 27 02:16:11 2022] POST /compute/v2.1/flavors => generated 409 bytes in 461 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Aug 27 02:16:12.900332 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi [None req-92c73149-2af0-44e5-bc98-9a8a1fe34dc4 admin admin] Calling method '>' {{(pid=105769) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:16:12.901393 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: INFO nova.api.openstack.requestlog [None req-92c73149-2af0-44e5-bc98-9a8a1fe34dc4 admin admin] 10.222.0.52 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001790 Aug 27 02:16:12.901815 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: [pid: 105769|app: 0|req: 31/61] 10.222.0.52 () {56 vars in 1074 bytes} [Sat Aug 27 02:16:12 2022] GET /compute/v2.1 => generated 388 bytes in 3 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 27 02:16:13.493277 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi [None req-92485520-647c-4798-8f9e-e8005ee68a7b admin admin] Calling method '>' {{(pid=105770) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:16:13.515760 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: INFO nova.api.openstack.requestlog [None req-92485520-647c-4798-8f9e-e8005ee68a7b admin admin] 10.222.0.52 "GET /compute/v2.1/flavors/detail" status: 200 len: 5777 microversion: 2.90 time: 0.468979 Aug 27 02:16:13.516326 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: [pid: 105770|app: 0|req: 31/62] 10.222.0.52 () {60 vars in 1202 bytes} [Sat Aug 27 02:16:13 2022] GET /compute/v2.1/flavors/detail => generated 5777 bytes in 470 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 27 02:17:11.491917 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi [None req-92c73149-2af0-44e5-bc98-9a8a1fe34dc4 admin admin] Calling method '>' {{(pid=105769) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:17:11.492929 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: INFO nova.api.openstack.requestlog [None req-92c73149-2af0-44e5-bc98-9a8a1fe34dc4 admin admin] 10.222.0.52 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001590 Aug 27 02:17:11.493456 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: [pid: 105769|app: 0|req: 32/63] 10.222.0.52 () {54 vars in 931 bytes} [Sat Aug 27 02:17:11 2022] GET /compute/v2.1 => generated 388 bytes in 3 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 27 02:17:11.505009 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG dbcounter [-] [105770] Writing DB stats nova_api:INSERT=5,nova_api:SELECT=28 {{(pid=105770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 27 02:17:12.018916 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi [None req-77e7f6d3-6dda-4bd8-b5f9-4a8d7355d90a admin admin] Calling method '>' {{(pid=105770) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:17:12.040763 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: INFO nova.api.openstack.requestlog [None req-77e7f6d3-6dda-4bd8-b5f9-4a8d7355d90a admin admin] 10.222.0.52 "GET /compute/v2.1/flavors/detail?is_public=True" status: 200 len: 5791 microversion: 2.61 time: 0.540377 Aug 27 02:17:12.041352 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: [pid: 105770|app: 0|req: 32/64] 10.222.0.52 () {60 vars in 1292 bytes} [Sat Aug 27 02:17:11 2022] GET /compute/v2.1/flavors/detail?is_public=True => generated 5791 bytes in 542 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Aug 27 02:17:12.065029 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG dbcounter [-] [105769] Writing DB stats nova_api:SELECT=14 {{(pid=105769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 27 02:17:12.068864 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi [None req-597a102b-0491-4bb8-9819-70bbe7a2d0a6 admin admin] Calling method '>' {{(pid=105769) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:17:12.078518 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: INFO nova.api.openstack.requestlog [None req-597a102b-0491-4bb8-9819-70bbe7a2d0a6 admin admin] 10.222.0.52 "GET /compute/v2.1/flavors/42/os-extra_specs" status: 200 len: 19 microversion: 2.61 time: 0.014693 Aug 27 02:17:12.078518 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: [pid: 105769|app: 0|req: 33/65] 10.222.0.52 () {60 vars in 1283 bytes} [Sat Aug 27 02:17:12 2022] GET /compute/v2.1/flavors/42/os-extra_specs => generated 19 bytes in 16 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:17:12.091747 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi [None req-71f247db-de5d-4b78-abd2-7edbe076f278 admin admin] Calling method '>' {{(pid=105770) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:17:12.100674 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: INFO nova.api.openstack.requestlog [None req-71f247db-de5d-4b78-abd2-7edbe076f278 admin admin] 10.222.0.52 "GET /compute/v2.1/flavors/451/os-extra_specs" status: 200 len: 19 microversion: 2.61 time: 0.012650 Aug 27 02:17:12.101255 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: [pid: 105770|app: 0|req: 33/66] 10.222.0.52 () {60 vars in 1286 bytes} [Sat Aug 27 02:17:12 2022] GET /compute/v2.1/flavors/451/os-extra_specs => generated 19 bytes in 14 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:17:12.113567 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi [None req-7d8159ec-405c-4213-ae71-e5067d5c0d7d admin admin] Calling method '>' {{(pid=105769) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:17:12.121842 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: INFO nova.api.openstack.requestlog [None req-7d8159ec-405c-4213-ae71-e5067d5c0d7d admin admin] 10.222.0.52 "GET /compute/v2.1/flavors/452/os-extra_specs" status: 200 len: 19 microversion: 2.61 time: 0.011906 Aug 27 02:17:12.122442 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: [pid: 105769|app: 0|req: 34/67] 10.222.0.52 () {60 vars in 1286 bytes} [Sat Aug 27 02:17:12 2022] GET /compute/v2.1/flavors/452/os-extra_specs => generated 19 bytes in 13 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:17:12.138213 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi [None req-257adfeb-ada3-4889-95cb-033a413ce263 admin admin] Calling method '>' {{(pid=105770) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:17:12.150240 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: INFO nova.api.openstack.requestlog [None req-257adfeb-ada3-4889-95cb-033a413ce263 admin admin] 10.222.0.52 "GET /compute/v2.1/flavors/84/os-extra_specs" status: 200 len: 19 microversion: 2.61 time: 0.017919 Aug 27 02:17:12.151121 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: [pid: 105770|app: 0|req: 34/68] 10.222.0.52 () {60 vars in 1283 bytes} [Sat Aug 27 02:17:12 2022] GET /compute/v2.1/flavors/84/os-extra_specs => generated 19 bytes in 20 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:20:21.513585 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi [None req-7d8159ec-405c-4213-ae71-e5067d5c0d7d admin admin] Calling method '>' {{(pid=105769) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:20:21.514521 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: INFO nova.api.openstack.requestlog [None req-7d8159ec-405c-4213-ae71-e5067d5c0d7d admin admin] 10.222.0.52 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001489 Aug 27 02:20:21.515267 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: [pid: 105769|app: 0|req: 35/69] 10.222.0.52 () {56 vars in 1074 bytes} [Sat Aug 27 02:20:21 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Aug 27 02:20:21.653967 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG dbcounter [-] [105770] Writing DB stats nova_api:SELECT=6 {{(pid=105770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 27 02:20:21.856373 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi [None req-50d75b25-dcbb-484e-a8bc-059faec72c3d admin admin] Calling method '>' {{(pid=105770) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:20:21.860404 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG oslo_concurrency.lockutils [None req-50d75b25-dcbb-484e-a8bc-059faec72c3d admin admin] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=105770) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:20:21.860870 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG oslo_concurrency.lockutils [None req-50d75b25-dcbb-484e-a8bc-059faec72c3d admin admin] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=105770) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:20:21.861314 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG oslo_concurrency.lockutils [None req-50d75b25-dcbb-484e-a8bc-059faec72c3d admin admin] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=105770) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:20:21.862012 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG oslo_concurrency.lockutils [None req-50d75b25-dcbb-484e-a8bc-059faec72c3d admin admin] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=105770) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:20:21.862438 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG oslo_concurrency.lockutils [None req-50d75b25-dcbb-484e-a8bc-059faec72c3d admin admin] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=105770) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:20:21.862838 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG oslo_concurrency.lockutils [None req-50d75b25-dcbb-484e-a8bc-059faec72c3d admin admin] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=105770) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:20:21.899182 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: INFO nova.api.openstack.requestlog [None req-50d75b25-dcbb-484e-a8bc-059faec72c3d admin admin] 10.222.0.52 "GET /compute/v2.1/os-services" status: 200 len: 791 microversion: 2.90 time: 0.246629 Aug 27 02:20:21.899766 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: [pid: 105770|app: 0|req: 35/70] 10.222.0.52 () {60 vars in 1193 bytes} [Sat Aug 27 02:20:21 2022] GET /compute/v2.1/os-services => generated 791 bytes in 248 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Aug 27 02:20:22.770538 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi [None req-7d8159ec-405c-4213-ae71-e5067d5c0d7d admin admin] Calling method '>' {{(pid=105769) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:20:22.771544 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: INFO nova.api.openstack.requestlog [None req-7d8159ec-405c-4213-ae71-e5067d5c0d7d admin admin] 10.222.0.52 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001470 Aug 27 02:20:22.771937 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: [pid: 105769|app: 0|req: 36/71] 10.222.0.52 () {56 vars in 1074 bytes} [Sat Aug 27 02:20:22 2022] GET /compute/v2.1 => generated 388 bytes in 3 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 27 02:20:22.988884 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi [None req-9217dd79-8b95-43dc-a29a-5293cc8e4d67 admin admin] Calling method '>' {{(pid=105770) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:20:22.992636 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG oslo_concurrency.lockutils [None req-9217dd79-8b95-43dc-a29a-5293cc8e4d67 admin admin] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=105770) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:20:22.992878 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG oslo_concurrency.lockutils [None req-9217dd79-8b95-43dc-a29a-5293cc8e4d67 admin admin] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=105770) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:20:22.993312 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG oslo_concurrency.lockutils [None req-9217dd79-8b95-43dc-a29a-5293cc8e4d67 admin admin] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=105770) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:20:22.994026 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG oslo_concurrency.lockutils [None req-9217dd79-8b95-43dc-a29a-5293cc8e4d67 admin admin] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=105770) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:20:22.994465 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG oslo_concurrency.lockutils [None req-9217dd79-8b95-43dc-a29a-5293cc8e4d67 admin admin] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=105770) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:20:22.994874 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG oslo_concurrency.lockutils [None req-9217dd79-8b95-43dc-a29a-5293cc8e4d67 admin admin] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=105770) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:20:23.015280 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: INFO nova.api.openstack.requestlog [None req-9217dd79-8b95-43dc-a29a-5293cc8e4d67 admin admin] 10.222.0.52 "GET /compute/v2.1/os-services" status: 200 len: 791 microversion: 2.90 time: 0.093661 Aug 27 02:20:23.015844 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: [pid: 105770|app: 0|req: 36/72] 10.222.0.52 () {60 vars in 1193 bytes} [Sat Aug 27 02:20:22 2022] GET /compute/v2.1/os-services => generated 791 bytes in 95 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Aug 27 02:20:34.321321 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi [None req-7d8159ec-405c-4213-ae71-e5067d5c0d7d admin admin] Calling method '>' {{(pid=105769) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:20:34.322461 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: INFO nova.api.openstack.requestlog [None req-7d8159ec-405c-4213-ae71-e5067d5c0d7d admin admin] 10.222.0.52 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001546 Aug 27 02:20:34.322793 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: [pid: 105769|app: 0|req: 37/73] 10.222.0.52 () {56 vars in 1074 bytes} [Sat Aug 27 02:20:34 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 27 02:20:34.472031 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG dbcounter [-] [105770] Writing DB stats nova_cell0:SELECT=4 {{(pid=105770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 27 02:20:34.472692 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG dbcounter [-] [105770] Writing DB stats nova_cell1:SELECT=4 {{(pid=105770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 27 02:20:34.473350 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG dbcounter [-] [105770] Writing DB stats nova_api:SELECT=8 {{(pid=105770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 27 02:20:34.525694 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi [None req-29d62b8e-227b-4b6d-8137-459f079fae68 admin admin] Calling method '>' {{(pid=105770) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:20:34.531759 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG oslo_concurrency.lockutils [None req-29d62b8e-227b-4b6d-8137-459f079fae68 admin admin] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=105770) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:20:34.532635 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG oslo_concurrency.lockutils [None req-29d62b8e-227b-4b6d-8137-459f079fae68 admin admin] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=105770) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:20:34.533147 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG oslo_concurrency.lockutils [None req-29d62b8e-227b-4b6d-8137-459f079fae68 admin admin] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=105770) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:20:34.533875 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG oslo_concurrency.lockutils [None req-29d62b8e-227b-4b6d-8137-459f079fae68 admin admin] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=105770) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:20:34.534313 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG oslo_concurrency.lockutils [None req-29d62b8e-227b-4b6d-8137-459f079fae68 admin admin] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=105770) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:20:34.534715 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG oslo_concurrency.lockutils [None req-29d62b8e-227b-4b6d-8137-459f079fae68 admin admin] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=105770) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:20:34.560638 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: INFO nova.api.openstack.requestlog [None req-29d62b8e-227b-4b6d-8137-459f079fae68 admin admin] 10.222.0.52 "GET /compute/v2.1/os-services" status: 200 len: 1011 microversion: 2.90 time: 0.095901 Aug 27 02:20:34.561270 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: [pid: 105770|app: 0|req: 37/74] 10.222.0.52 () {60 vars in 1193 bytes} [Sat Aug 27 02:20:34 2022] GET /compute/v2.1/os-services => generated 1011 bytes in 97 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 27 02:20:35.425532 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: DEBUG nova.api.openstack.wsgi [None req-7d8159ec-405c-4213-ae71-e5067d5c0d7d admin admin] Calling method '>' {{(pid=105769) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:20:35.428260 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: INFO nova.api.openstack.requestlog [None req-7d8159ec-405c-4213-ae71-e5067d5c0d7d admin admin] 10.222.0.52 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001499 Aug 27 02:20:35.428260 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: [pid: 105769|app: 0|req: 38/75] 10.222.0.52 () {56 vars in 1074 bytes} [Sat Aug 27 02:20:35 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 27 02:20:35.625344 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG nova.api.openstack.wsgi [None req-d3593b28-bf6d-4798-bb89-5a0d67149f7d admin admin] Calling method '>' {{(pid=105770) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:20:35.628819 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG oslo_concurrency.lockutils [None req-d3593b28-bf6d-4798-bb89-5a0d67149f7d admin admin] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=105770) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:20:35.629308 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG oslo_concurrency.lockutils [None req-d3593b28-bf6d-4798-bb89-5a0d67149f7d admin admin] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=105770) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:20:35.629724 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG oslo_concurrency.lockutils [None req-d3593b28-bf6d-4798-bb89-5a0d67149f7d admin admin] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=105770) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:20:35.630445 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG oslo_concurrency.lockutils [None req-d3593b28-bf6d-4798-bb89-5a0d67149f7d admin admin] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=105770) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:20:35.630872 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG oslo_concurrency.lockutils [None req-d3593b28-bf6d-4798-bb89-5a0d67149f7d admin admin] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=105770) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:20:35.631277 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: DEBUG oslo_concurrency.lockutils [None req-d3593b28-bf6d-4798-bb89-5a0d67149f7d admin admin] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=105770) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:20:35.653622 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: INFO nova.api.openstack.requestlog [None req-d3593b28-bf6d-4798-bb89-5a0d67149f7d admin admin] 10.222.0.52 "GET /compute/v2.1/os-services" status: 200 len: 1011 microversion: 2.90 time: 0.085190 Aug 27 02:20:35.654872 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: [pid: 105770|app: 0|req: 38/76] 10.222.0.52 () {60 vars in 1193 bytes} [Sat Aug 27 02:20:35 2022] GET /compute/v2.1/os-services => generated 1011 bytes in 87 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 27 02:21:34.716856 dv-cinder-846303-3-iscsi systemd[1]: Stopping Devstack devstack@n-api.service... Aug 27 02:21:34.717501 dv-cinder-846303-3-iscsi devstack@n-api.service[105768]: Sat Aug 27 02:21:34 2022 - graceful shutdown triggered... Aug 27 02:21:34.717984 dv-cinder-846303-3-iscsi devstack@n-api.service[105770]: Gracefully killing worker 2 (pid: 105770)... Aug 27 02:21:34.718556 dv-cinder-846303-3-iscsi devstack@n-api.service[105769]: Gracefully killing worker 1 (pid: 105769)... Aug 27 02:21:35.718658 dv-cinder-846303-3-iscsi devstack@n-api.service[105768]: worker 1 buried after 1 seconds Aug 27 02:21:35.719056 dv-cinder-846303-3-iscsi devstack@n-api.service[105768]: worker 2 buried after 1 seconds Aug 27 02:21:35.719056 dv-cinder-846303-3-iscsi devstack@n-api.service[105768]: goodbye to uWSGI. Aug 27 02:21:35.720498 dv-cinder-846303-3-iscsi systemd[1]: devstack@n-api.service: Succeeded. Aug 27 02:21:35.721148 dv-cinder-846303-3-iscsi systemd[1]: Stopped Devstack devstack@n-api.service. Aug 27 02:21:35.724143 dv-cinder-846303-3-iscsi systemd[1]: Starting Devstack devstack@n-api.service... Aug 27 02:21:35.736384 dv-cinder-846303-3-iscsi devstack@n-api.service[120846]: [uWSGI] getting INI configuration from /etc/nova/nova-api-uwsgi.ini Aug 27 02:21:35.739122 dv-cinder-846303-3-iscsi devstack@n-api.service[120846]: *** Starting uWSGI 2.0.18-debian (64bit) on [Sat Aug 27 02:21:35 2022] *** Aug 27 02:21:35.739122 dv-cinder-846303-3-iscsi devstack@n-api.service[120846]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Aug 27 02:21:35.739122 dv-cinder-846303-3-iscsi devstack@n-api.service[120846]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Aug 27 02:21:35.739122 dv-cinder-846303-3-iscsi devstack@n-api.service[120846]: nodename: dv-cinder-846303-3-iscsi Aug 27 02:21:35.739122 dv-cinder-846303-3-iscsi devstack@n-api.service[120846]: machine: x86_64 Aug 27 02:21:35.739122 dv-cinder-846303-3-iscsi devstack@n-api.service[120846]: clock source: unix Aug 27 02:21:35.739122 dv-cinder-846303-3-iscsi devstack@n-api.service[120846]: pcre jit disabled Aug 27 02:21:35.739122 dv-cinder-846303-3-iscsi devstack@n-api.service[120846]: detected number of CPU cores: 4 Aug 27 02:21:35.739794 dv-cinder-846303-3-iscsi devstack@n-api.service[120846]: current working directory: / Aug 27 02:21:35.739794 dv-cinder-846303-3-iscsi devstack@n-api.service[120846]: detected binary path: /usr/bin/uwsgi-core Aug 27 02:21:35.739794 dv-cinder-846303-3-iscsi devstack@n-api.service[120846]: your processes number limit is 47680 Aug 27 02:21:35.739794 dv-cinder-846303-3-iscsi devstack@n-api.service[120846]: your memory page size is 4096 bytes Aug 27 02:21:35.739794 dv-cinder-846303-3-iscsi devstack@n-api.service[120846]: detected max file descriptor number: 2048 Aug 27 02:21:35.739794 dv-cinder-846303-3-iscsi devstack@n-api.service[120846]: lock engine: pthread robust mutexes Aug 27 02:21:35.739794 dv-cinder-846303-3-iscsi devstack@n-api.service[120846]: thunder lock: enabled Aug 27 02:21:35.739794 dv-cinder-846303-3-iscsi devstack@n-api.service[120846]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/nova-api-wsgi.socket fd 4 Aug 27 02:21:35.739794 dv-cinder-846303-3-iscsi devstack@n-api.service[120846]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Aug 27 02:21:35.792539 dv-cinder-846303-3-iscsi devstack@n-api.service[120846]: Python main interpreter initialized at 0x563e575eb9c0 Aug 27 02:21:35.792539 dv-cinder-846303-3-iscsi devstack@n-api.service[120846]: python threads support enabled Aug 27 02:21:35.792539 dv-cinder-846303-3-iscsi devstack@n-api.service[120846]: your server socket listen backlog is limited to 100 connections Aug 27 02:21:35.792539 dv-cinder-846303-3-iscsi devstack@n-api.service[120846]: your mercy for graceful operations on workers is 90 seconds Aug 27 02:21:35.793154 dv-cinder-846303-3-iscsi devstack@n-api.service[120846]: mapped 403077 bytes (393 KB) for 2 cores Aug 27 02:21:35.793272 dv-cinder-846303-3-iscsi devstack@n-api.service[120846]: *** Operational MODE: preforking *** Aug 27 02:21:35.793622 dv-cinder-846303-3-iscsi devstack@n-api.service[120846]: *** uWSGI is running in multiple interpreter mode *** Aug 27 02:21:35.793622 dv-cinder-846303-3-iscsi devstack@n-api.service[120846]: spawned uWSGI master process (pid: 120846) Aug 27 02:21:35.793901 dv-cinder-846303-3-iscsi systemd[1]: Started Devstack devstack@n-api.service. Aug 27 02:21:35.797221 dv-cinder-846303-3-iscsi devstack@n-api.service[120846]: spawned uWSGI worker 1 (pid: 120863, cores: 1) Aug 27 02:21:35.797221 dv-cinder-846303-3-iscsi devstack@n-api.service[120846]: spawned uWSGI worker 2 (pid: 120864, cores: 1) Aug 27 02:21:35.797221 dv-cinder-846303-3-iscsi devstack@n-api.service[120846]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Aug 27 02:21:36.482868 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: Modules with known eventlet monkey patching issues were imported prior to eventlet monkey patching: urllib3. This warning can usually be ignored if the caller is only importing and not executing nova code. Aug 27 02:21:36.536090 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: Modules with known eventlet monkey patching issues were imported prior to eventlet monkey patching: urllib3. This warning can usually be ignored if the caller is only importing and not executing nova code. Aug 27 02:21:37.684465 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 27 02:21:37.684465 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: from cryptography.utils import int_from_bytes Aug 27 02:21:37.686131 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 27 02:21:37.686131 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: from cryptography.utils import int_from_bytes Aug 27 02:21:37.707550 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 27 02:21:37.707550 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: from cryptography.utils import int_from_bytes Aug 27 02:21:37.708181 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 27 02:21:37.708181 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: from cryptography.utils import int_from_bytes Aug 27 02:21:40.240644 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] ******************************************************************************** {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Aug 27 02:21:40.241648 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] Configuration options gathered from: {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Aug 27 02:21:40.241822 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] command line args: [] {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Aug 27 02:21:40.242205 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] config files: ['/etc/nova/api-paste.ini', '/etc/nova/nova.conf'] {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Aug 27 02:21:40.242479 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] ================================================================================ {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Aug 27 02:21:40.242818 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] allow_resize_to_same_host = True {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:21:40.243279 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] arq_binding_timeout = 300 {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:21:40.243615 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] block_device_allocate_retries = 60 {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:21:40.243942 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] block_device_allocate_retries_interval = 3 {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:21:40.244451 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] cert = self.pem {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:21:40.244770 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] compute_driver = None {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:21:40.245101 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] compute_monitors = [] {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:21:40.245457 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] config_dir = [] {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:21:40.245786 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] config_drive_format = iso9660 {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:21:40.246043 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] config_file = ['/etc/nova/api-paste.ini', '/etc/nova/nova.conf'] {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:21:40.246362 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] config_source = [] {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:21:40.246689 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] console_host = dv-cinder-846303-3-iscsi {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:21:40.247015 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] control_exchange = nova {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:21:40.247325 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] cpu_allocation_ratio = None {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:21:40.247637 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] daemon = False {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:21:40.247961 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] debug = True {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:21:40.248266 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] default_access_ip_network_name = None {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:21:40.248594 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] default_availability_zone = nova {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:21:40.248903 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] default_ephemeral_format = None {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:21:40.249447 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'glanceclient=WARN', 'oslo.privsep.daemon=INFO'] {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:21:40.251040 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] default_schedule_zone = None {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:21:40.251400 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] disk_allocation_ratio = None {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:21:40.251772 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] enable_new_services = True {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:21:40.252133 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] enabled_apis = ['osapi_compute'] {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:21:40.252550 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] enabled_ssl_apis = [] {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:21:40.252875 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] flat_injected = False {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:21:40.253493 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] force_config_drive = True {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:21:40.253875 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] force_raw_images = True {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:21:40.254218 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] graceful_shutdown_timeout = 5 {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:21:40.254535 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] heal_instance_info_cache_interval = 60 {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:21:40.255132 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] host = dv-cinder-846303-3-iscsi {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:21:40.255616 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] initial_cpu_allocation_ratio = 16.0 {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:21:40.256094 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] initial_disk_allocation_ratio = 1.0 {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:21:40.256571 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] initial_ram_allocation_ratio = 1.5 {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:21:40.257177 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] injected_network_template = /opt/stack/nova/nova/virt/interfaces.template {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:21:40.257660 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] instance_build_timeout = 0 {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:21:40.258134 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] instance_delete_interval = 300 {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:21:40.258620 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] instance_format = [instance: %(uuid)s] {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:21:40.259151 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] instance_name_template = instance-%08x {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:21:40.259626 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] instance_usage_audit = False {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:21:40.260109 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] instance_usage_audit_period = month {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:21:40.260579 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:21:40.261060 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] instances_path = /opt/stack/data/nova/instances {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:21:40.261550 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] internal_service_availability_zone = internal {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:21:40.262014 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] key = None {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:21:40.262479 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] live_migration_retry_count = 30 {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:21:40.262940 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] log_config_append = None {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:21:40.263406 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:21:40.263881 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] log_dir = None {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:21:40.264346 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] log_file = None {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:21:40.264748 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] log_options = True {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:21:40.265326 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] log_rotate_interval = 1 {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:21:40.265823 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] log_rotate_interval_type = days {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:21:40.266310 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] log_rotation_type = none {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:21:40.266815 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:21:40.267065 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:21:40.267492 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:21:40.267823 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:21:40.268152 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:21:40.268470 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] long_rpc_timeout = 1800 {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:21:40.268785 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_builds = 10 {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:21:40.269098 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_live_migrations = 1 {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:21:40.269445 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_snapshots = 5 {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:21:40.269927 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] max_local_block_devices = 3 {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:21:40.270456 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] max_logfile_count = 30 {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:21:40.270772 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] max_logfile_size_mb = 200 {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:21:40.271084 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] maximum_instance_delete_attempts = 5 {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:21:40.271410 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] metadata_listen = 0.0.0.0 {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:21:40.271731 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] metadata_listen_port = 8775 {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:21:40.272060 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] metadata_workers = 2 {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:21:40.272388 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] migrate_max_retries = -1 {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:21:40.272690 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] mkisofs_cmd = genisoimage {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:21:40.273097 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] my_block_storage_ip = 10.222.0.52 {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:21:40.273368 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] my_ip = 10.222.0.52 {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:21:40.273848 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] network_allocate_retries = 0 {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:21:40.274376 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] non_inheritable_image_properties = ['cache_in_nova', 'bittorrent'] {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:21:40.275139 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_listen = 0.0.0.0 {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:21:40.275461 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_listen_port = 8774 {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:21:40.275786 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_unique_server_name_scope = {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:21:40.276116 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_workers = 2 {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:21:40.276425 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] password_length = 12 {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:21:40.276737 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] periodic_enable = True {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:21:40.277048 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] periodic_fuzzy_delay = 60 {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:21:40.277398 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] pointer_model = usbtablet {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:21:40.277748 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] preallocate_images = none {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:21:40.278041 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] publish_errors = False {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:21:40.278450 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] pybasedir = /opt/stack/nova {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:21:40.278890 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] ram_allocation_ratio = None {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:21:40.279349 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_burst = 0 {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:21:40.279825 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_except_level = CRITICAL {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:21:40.280292 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_interval = 0 {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:21:40.280754 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] reboot_timeout = 0 {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:21:40.281069 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] reclaim_instance_interval = 0 {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:21:40.281415 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] record = None {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:21:40.281724 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] report_interval = 10 {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:21:40.282036 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] rescue_timeout = 0 {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:21:40.282346 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_cpus = 0 {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:21:40.282656 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_disk_mb = 0 {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:21:40.282964 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_memory_mb = 512 {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:21:40.283365 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] reserved_huge_pages = None {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:21:40.283677 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] resize_confirm_window = 0 {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:21:40.283984 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] resize_fs_using_block_device = False {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:21:40.284326 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] resume_guests_state_on_host_boot = False {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:21:40.284657 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] rootwrap_config = /etc/nova/rootwrap.conf {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:21:40.284976 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_action = reap {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:21:40.285314 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_poll_interval = 1800 {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:21:40.285628 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_timeout = 0 {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:21:40.285941 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] scheduler_instance_sync_interval = 120 {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:21:40.286246 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] service_down_time = 60 {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:21:40.286570 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] servicegroup_driver = db {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:21:40.286865 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] shelved_offload_time = 0 {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:21:40.287174 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] shelved_poll_interval = 3600 {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:21:40.287507 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] shutdown_timeout = 0 {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:21:40.287813 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] source_is_ipv6 = False {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:21:40.288118 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] ssl_only = False {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:21:40.288439 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] state_path = /opt/stack/data/nova {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:21:40.288749 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] sync_power_state_interval = 600 {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:21:40.289058 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] sync_power_state_pool_size = 1000 {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:21:40.289410 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] syslog_log_facility = LOG_USER {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:21:40.289715 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] tempdir = None {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:21:40.290088 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] timeout_nbd = 10 {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:21:40.290378 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] transport_url = **** {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:21:40.290726 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] update_resources_interval = 0 {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:21:40.290990 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] use_cow_images = True {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:21:40.291864 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] use_eventlog = False {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:21:40.291864 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] use_journal = False {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:21:40.292481 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] use_json = False {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:21:40.292481 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] use_rootwrap_daemon = False {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:21:40.292972 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] use_stderr = False {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:21:40.292972 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] use_syslog = False {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:21:40.293448 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] vcpu_pin_set = None {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:21:40.293693 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] vif_plugging_is_fatal = True {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:21:40.293927 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] vif_plugging_timeout = 300 {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:21:40.294148 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] virt_mkfs = [] {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:21:40.295709 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] volume_usage_poll_interval = 0 {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:21:40.295709 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] watch_log_file = False {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:21:40.295709 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] web = /usr/share/spice-html5 {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:21:40.295709 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] oslo_concurrency.disable_process_locking = False {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.296767 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] oslo_concurrency.lock_path = /opt/stack/data/nova {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.296767 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] api.auth_strategy = keystone {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.296767 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] api.compute_link_prefix = None {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.296767 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] api.config_drive_skip_versions = 1.0 2007-01-19 2007-03-01 2007-08-29 2007-10-10 2007-12-15 2008-02-01 2008-09-01 {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.297699 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] api.dhcp_domain = novalocal {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.297699 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] api.enable_instance_password = True {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.297699 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] api.glance_link_prefix = None {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.298384 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_cells_batch_fixed_size = 100 {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.298384 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_cells_batch_strategy = distributed {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.298901 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_per_project_cells = False {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.299174 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] api.list_records_by_skipping_down_cells = True {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.299430 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] api.local_metadata_per_cell = False {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.299677 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] api.max_limit = 1000 {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.299938 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] api.metadata_cache_expiration = 15 {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.300184 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] api.neutron_default_tenant_id = default {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.300608 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] api.use_forwarded_for = False {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.300874 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] api.use_neutron_default_nets = False {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.301113 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_connect_timeout = 5 {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.301423 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_failure_fatal = False {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.302863 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_read_timeout = 5 {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.302863 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_ssl_certfile = {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.302863 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_targets = [] {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.302863 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_jsonfile_path = None {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.303822 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_providers = ['StaticJSON'] {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.303822 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] cache.backend = dogpile.cache.memcached {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.303822 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] cache.backend_argument = **** {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.304488 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] cache.config_prefix = cache.oslo {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.304488 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] cache.dead_timeout = 60.0 {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.305026 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] cache.debug_cache_backend = False {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.305026 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] cache.enable_retry_client = False {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.305992 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] cache.enable_socket_keepalive = False {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.305992 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] cache.enabled = True {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.305992 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] cache.expiration_time = 600 {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.306745 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] cache.hashclient_retry_attempts = 2 {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.306745 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] cache.hashclient_retry_delay = 1.0 {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.307216 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_dead_retry = 300 {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.307216 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_password = {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.307688 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.307931 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.308164 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_maxsize = 10 {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.308464 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_unused_timeout = 60 {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.308781 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_sasl_enabled = False {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.309325 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_servers = ['localhost:11211'] {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.309861 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_socket_timeout = 1.0 {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.310397 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_username = {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.310939 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] cache.proxies = [] {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.311511 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] cache.retry_attempts = 2 {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.312804 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] cache.retry_delay = 0.0 {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.312804 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_count = 1 {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.313527 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_idle = 1 {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.313527 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_interval = 1 {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.313981 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_allowed_ciphers = None {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.314216 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_cafile = None {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.314624 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_certfile = None {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.315041 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_enabled = False {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.315434 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_keyfile = None {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.315845 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] cinder.auth_section = None {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.316438 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] cinder.auth_type = password {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.316948 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] cinder.cafile = None {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.317571 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] cinder.catalog_info = volumev3::publicURL {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.318113 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] cinder.certfile = None {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.318686 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] cinder.collect_timing = False {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.319162 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] cinder.cross_az_attach = True {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.319708 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] cinder.debug = False {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.320256 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] cinder.endpoint_template = None {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.320825 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] cinder.http_retries = 3 {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.321446 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] cinder.insecure = False {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.322010 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] cinder.keyfile = None {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.323055 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] cinder.os_region_name = RegionOne {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.323330 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] cinder.split_loggers = False {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.323741 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] cinder.timeout = None {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.323968 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] compute.consecutive_build_service_disable_threshold = 10 {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.324423 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] compute.cpu_dedicated_set = None {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.324781 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] compute.cpu_shared_set = None {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.325637 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] compute.image_type_exclude_list = [] {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.326085 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] compute.live_migration_wait_for_vif_plug = True {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.326416 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] compute.max_concurrent_disk_ops = 0 {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.326851 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] compute.max_disk_devices_to_attach = -1 {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.327087 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] compute.packing_host_numa_cells_allocation_strategy = False {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.327403 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] compute.provider_config_location = /etc/nova/provider_config/ {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.327728 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] compute.resource_provider_association_refresh = 300 {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.328050 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] compute.shutdown_retry_interval = 10 {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.328402 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] conductor.workers = 2 {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.328739 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] console.allowed_origins = [] {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.329931 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] console.ssl_ciphers = None {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.329931 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] console.ssl_minimum_version = default {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.329931 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] consoleauth.token_ttl = 600 {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.330657 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.cafile = None {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.330657 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.certfile = None {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.330657 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.collect_timing = False {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.331568 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.connect_retries = None {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.331568 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.connect_retry_delay = None {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.331568 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.endpoint_override = None {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.332242 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.insecure = False {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.332242 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.keyfile = None {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.333505 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.max_version = None {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.333505 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.min_version = None {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.333505 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.region_name = None {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.333505 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.service_name = None {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.334669 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.service_type = accelerator {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.334669 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.split_loggers = False {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.334669 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.status_code_retries = None {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.335362 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.status_code_retry_delay = None {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.335362 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.timeout = None {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.335362 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.valid_interfaces = ['internal', 'public'] {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.336126 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.version = None {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.336126 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] database.backend = sqlalchemy {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.336704 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] database.connection = **** {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.336704 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_debug = 0 {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.337563 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_parameters = {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.337563 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_recycle_time = 3600 {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.338014 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_trace = False {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.338014 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] database.db_inc_retry_interval = True {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.338465 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] database.db_max_retries = 20 {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.338705 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] database.db_max_retry_interval = 10 {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.338933 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] database.db_retry_interval = 1 {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.339170 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] database.max_overflow = 50 {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.339496 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] database.max_pool_size = 5 {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.339820 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] database.max_retries = 10 {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.340136 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] database.mysql_enable_ndb = False {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.340466 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] database.mysql_sql_mode = TRADITIONAL {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.340780 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] database.pool_timeout = None {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.341082 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] database.retry_interval = 10 {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.341442 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] database.slave_connection = **** {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.341861 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] database.sqlite_synchronous = True {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.342128 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] api_database.backend = sqlalchemy {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.342475 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection = **** {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.342891 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_debug = 0 {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.343779 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_parameters = {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.343779 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_recycle_time = 3600 {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.343779 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_trace = False {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.344699 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_inc_retry_interval = True {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.344699 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_max_retries = 20 {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.344699 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_max_retry_interval = 10 {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.345455 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_retry_interval = 1 {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.345455 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_overflow = 50 {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.345455 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_pool_size = 5 {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.346136 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_retries = 10 {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.346136 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] api_database.mysql_enable_ndb = False {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.346794 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] api_database.mysql_sql_mode = TRADITIONAL {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.346794 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] api_database.pool_timeout = None {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.347457 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] api_database.retry_interval = 10 {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.347457 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] api_database.slave_connection = **** {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.347934 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] api_database.sqlite_synchronous = True {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.347934 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] devices.enabled_mdev_types = [] {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.348395 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.cipher = aes-xts-plain64 {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.348684 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.enabled = False {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.348912 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.key_size = 512 {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.349174 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] glance.api_servers = None {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.349460 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] glance.cafile = None {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.349771 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] glance.certfile = None {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.350081 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] glance.collect_timing = False {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.350386 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] glance.connect_retries = None {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.351636 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] glance.connect_retry_delay = None {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.351636 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] glance.debug = False {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.351636 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] glance.default_trusted_certificate_ids = [] {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.352533 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] glance.enable_certificate_validation = False {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.352533 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] glance.enable_rbd_download = False {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.353242 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] glance.endpoint_override = None {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.353242 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] glance.insecure = False {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.354509 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] glance.keyfile = None {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.354509 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] glance.max_version = None {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.354509 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] glance.min_version = None {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.354509 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] glance.num_retries = 3 {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.355637 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_ceph_conf = {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.355637 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_connect_timeout = 5 {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.355637 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_pool = {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.356337 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_user = {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.356337 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] glance.region_name = None {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.356337 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] glance.service_name = None {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.357524 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] glance.service_type = image {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.357524 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] glance.split_loggers = False {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.357524 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] glance.status_code_retries = None {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.358399 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] glance.status_code_retry_delay = None {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.358399 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] glance.timeout = None {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.358399 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] glance.valid_interfaces = ['internal', 'public'] {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.359142 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] glance.verify_glance_signatures = False {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.359142 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] glance.version = None {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.359596 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] guestfs.debug = False {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.359596 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.config_drive_cdrom = False {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.360070 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.config_drive_inject_password = False {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.360307 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.dynamic_memory_ratio = 1.0 {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.360551 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.enable_instance_metrics_collection = False {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.360793 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.enable_remotefx = False {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.361093 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.instances_path_share = {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.361487 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.iscsi_initiator_list = [] {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.361735 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.limit_cpu_features = False {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.362018 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.mounted_disk_query_retry_count = 10 {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.362434 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.mounted_disk_query_retry_interval = 5 {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.362696 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.power_state_check_timeframe = 60 {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.362940 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.power_state_event_polling_interval = 2 {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.363233 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.qemu_img_cmd = qemu-img.exe {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.363641 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.use_multipath_io = False {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.364515 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.volume_attach_retry_count = 10 {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.364515 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.volume_attach_retry_interval = 5 {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.364515 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.vswitch_name = None {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.365478 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.wait_soft_reboot_seconds = 60 {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.365478 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] mks.enabled = False {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.365954 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] mks.mksproxy_base_url = http://127.0.0.1:6090/ {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.366214 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.manager_interval = 2400 {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.366471 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.precache_concurrency = 1 {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.366729 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_base_images = True {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.367013 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_original_minimum_age_seconds = 86400 {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.367430 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_resized_minimum_age_seconds = 3600 {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.367696 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.subdirectory_name = _base {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.367979 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] ironic.api_max_retries = 60 {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.368480 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] ironic.api_retry_interval = 2 {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.368739 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] ironic.auth_section = None {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.368986 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] ironic.auth_type = None {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.369383 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] ironic.cafile = None {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.369699 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] ironic.certfile = None {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.370015 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] ironic.collect_timing = False {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.370482 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] ironic.connect_retries = None {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.370979 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] ironic.connect_retry_delay = None {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.371509 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] ironic.endpoint_override = None {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.372046 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] ironic.insecure = False {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.372582 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] ironic.keyfile = None {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.373153 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] ironic.max_version = None {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.373709 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] ironic.min_version = None {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.374166 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] ironic.partition_key = None {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.374866 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] ironic.peer_list = [] {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.375162 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] ironic.region_name = None {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.375728 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] ironic.serial_console_state_timeout = 10 {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.376279 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] ironic.service_name = None {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.376867 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] ironic.service_type = baremetal {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.377465 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] ironic.split_loggers = False {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.377968 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] ironic.status_code_retries = None {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.378592 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] ironic.status_code_retry_delay = None {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.379082 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] ironic.timeout = None {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.379706 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] ironic.valid_interfaces = ['internal', 'public'] {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.380248 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] ironic.version = None {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.380857 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] key_manager.backend = nova.keymgr.conf_key_mgr.ConfKeyManager {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.381498 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] key_manager.fixed_key = **** {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.382094 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] barbican.auth_endpoint = http://localhost/identity/v3 {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.382692 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_api_version = None {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.383199 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_endpoint = None {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.383936 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_endpoint_type = public {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.384366 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_region_name = None {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.385162 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] barbican.cafile = None {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.385722 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] barbican.certfile = None {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.386121 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] barbican.collect_timing = False {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.386682 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] barbican.insecure = False {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.387813 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] barbican.keyfile = None {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.387813 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] barbican.number_of_retries = 60 {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.387813 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] barbican.retry_delay = 1 {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.388700 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] barbican.send_service_user_token = False {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.388700 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] barbican.split_loggers = False {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.388700 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] barbican.timeout = None {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.389680 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] barbican.verify_ssl = True {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.389680 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] barbican.verify_ssl_path = None {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.389680 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.auth_section = None {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.390378 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.auth_type = None {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.390378 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.cafile = None {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.390889 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.certfile = None {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.390889 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.collect_timing = False {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.391357 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.insecure = False {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.391672 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.keyfile = None {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.391912 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.split_loggers = False {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.392164 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.timeout = None {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.392415 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] vault.approle_role_id = None {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.392658 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] vault.approle_secret_id = None {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.393068 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] vault.cafile = None {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.393338 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] vault.certfile = None {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.393610 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] vault.collect_timing = False {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.393863 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] vault.insecure = False {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.394275 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] vault.keyfile = None {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.394684 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] vault.kv_mountpoint = secret {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.395391 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] vault.kv_version = 2 {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.395391 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] vault.namespace = None {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.395867 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] vault.root_token_id = None {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.395867 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] vault.split_loggers = False {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.396649 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] vault.ssl_ca_crt_file = None {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.397103 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] vault.timeout = None {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.397386 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] vault.use_ssl = False {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.397765 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] vault.vault_url = http://127.0.0.1:8200 {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.398269 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] keystone.cafile = None {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.398533 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] keystone.certfile = None {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.398937 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] keystone.collect_timing = False {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.399482 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] keystone.connect_retries = None {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.399924 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] keystone.connect_retry_delay = None {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.400317 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] keystone.endpoint_override = None {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.400816 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] keystone.insecure = False {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.401245 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] keystone.keyfile = None {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.401579 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] keystone.max_version = None {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.401934 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] keystone.min_version = None {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.402193 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] keystone.region_name = None {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.402525 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] keystone.service_name = None {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.402873 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] keystone.service_type = identity {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.403306 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] keystone.split_loggers = False {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.403555 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] keystone.status_code_retries = None {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.403801 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] keystone.status_code_retry_delay = None {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.404148 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] keystone.timeout = None {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.404589 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] keystone.valid_interfaces = ['internal', 'public'] {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.405032 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] keystone.version = None {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.405758 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.connection_uri = {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.405758 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_mode = None {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.406256 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_model_extra_flags = [] {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.406256 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_models = [] {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.406723 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.device_detach_attempts = 8 {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.406944 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.device_detach_timeout = 20 {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.407346 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.disk_cachemodes = [] {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.407574 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.disk_prefix = None {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.407801 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.enabled_perf_events = [] {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.408080 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.file_backed_memory = 0 {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.408515 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.gid_maps = [] {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.408736 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.hw_disk_discard = None {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.409050 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.hw_machine_type = None {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.409530 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_ceph_conf = {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.409891 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_copy_poll_interval = 15 {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.410146 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_copy_timeout = 600 {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.410391 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_store_name = {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.410678 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_pool = rbd {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.411010 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_type = default {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.411327 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_volume_group = None {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.411650 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_key = False {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.412009 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_partition = -2 {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.412267 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_password = False {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.412719 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.iscsi_iface = None {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.413026 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.iser_use_multipath = False {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.413323 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_bandwidth = 0 {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.413715 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_completion_timeout = 800 {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.413978 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime = 500 {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.414273 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime_delay = 75 {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.414682 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime_steps = 10 {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.415165 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_inbound_addr = None {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.415869 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_permit_auto_converge = False {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.415869 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_permit_post_copy = False {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.416363 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_scheme = None {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.416607 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_timeout_action = abort {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.416902 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_tunnelled = False {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.417227 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_uri = None {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.417661 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_with_native_tls = False {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.418021 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.max_queues = None {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.418280 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.mem_stats_period_seconds = 10 {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.418526 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.nfs_mount_options = None {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.418909 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.nfs_mount_point_base = /opt/stack/data/nova/mnt {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.419341 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_aoe_discover_tries = 3 {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.419644 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_iser_scan_tries = 5 {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.420142 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_memory_encrypted_guests = None {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.420393 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_nvme_discover_tries = 5 {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.420640 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_pcie_ports = 0 {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.420889 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_volume_scan_tries = 5 {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.421182 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.pmem_namespaces = [] {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.421490 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.quobyte_client_cfg = None {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.421878 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.quobyte_mount_point_base = /opt/stack/data/nova/mnt {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.422186 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_connect_timeout = 5 {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.422504 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_destroy_volume_retries = 12 {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.422827 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_destroy_volume_retry_interval = 5 {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.423141 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_secret_uuid = None {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.423454 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_user = None {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.423896 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.realtime_scheduler_priority = 1 {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.424163 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.remote_filesystem_transport = ssh {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.424409 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_image_id = None {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.424865 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_kernel_id = None {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.425107 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_ramdisk_id = None {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.425600 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rng_dev_path = /dev/urandom {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.426300 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rx_queue_size = None {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.426300 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.smbfs_mount_options = {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.426772 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.smbfs_mount_point_base = /opt/stack/data/nova/mnt {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.426772 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshot_compression = False {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.427252 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshot_image_format = None {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.427514 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshots_directory = /opt/stack/data/nova/instances/snapshots {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.427960 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.sparse_logical_volumes = False {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.428207 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_enabled = False {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.428462 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_group = tss {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.428710 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_user = tss {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.429112 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.sysinfo_serial = unique {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.429404 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.tx_queue_size = None {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.429679 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.uid_maps = [] {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.429925 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.use_virtio_for_bridges = True {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.430382 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.virt_type = kvm {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.430776 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_clear = zero {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.431145 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_clear_size = 0 {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.431493 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_use_multipath = False {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.431983 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_cache_path = None {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.432310 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_log_path = /var/log/vstorage/%(cluster_name)s/nova.log.gz {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.432559 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_group = qemu {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.432897 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_opts = [] {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.433274 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_perms = 0770 {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.433714 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_point_base = /opt/stack/data/nova/mnt {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.434217 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_user = stack {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.434477 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.wait_soft_reboot_seconds = 120 {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.434736 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] neutron.auth_section = None {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.435156 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] neutron.auth_type = password {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.435425 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] neutron.cafile = None {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.435874 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] neutron.certfile = None {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.436595 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] neutron.collect_timing = False {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.436595 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] neutron.connect_retries = None {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.436595 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] neutron.connect_retry_delay = None {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.437353 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] neutron.default_floating_pool = public {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.437353 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] neutron.endpoint_override = None {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.437827 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] neutron.extension_sync_interval = 600 {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.438244 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] neutron.http_retries = 3 {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.438514 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] neutron.insecure = False {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.438753 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] neutron.keyfile = None {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.438990 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] neutron.max_version = None {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.439381 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] neutron.metadata_proxy_shared_secret = **** {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.439610 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] neutron.min_version = None {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.439833 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] neutron.ovs_bridge = br-int {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.440129 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] neutron.physnets = [] {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.440557 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] neutron.region_name = RegionOne {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.440809 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_metadata_proxy = True {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.441085 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_name = None {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.441441 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_type = network {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.441760 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] neutron.split_loggers = False {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.442076 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] neutron.status_code_retries = None {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.442372 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] neutron.status_code_retry_delay = None {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.442706 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] neutron.timeout = None {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.443032 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] neutron.valid_interfaces = ['internal', 'public'] {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.443346 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] neutron.version = None {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.443674 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] notifications.bdms_in_notifications = False {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.444041 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] notifications.default_level = INFO {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.444420 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] notifications.notification_format = unversioned {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.444715 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] notifications.notify_on_state_change = None {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.445082 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] notifications.versioned_notifications_topics = ['versioned_notifications'] {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.445496 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] pci.alias = [] {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.445776 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] pci.device_spec = [] {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.446128 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] ******************************************************************************** {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Aug 27 02:21:40.448565 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] placement.auth_section = None {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.448872 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] Configuration options gathered from: {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Aug 27 02:21:40.449154 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] placement.auth_type = password {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.449154 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] placement.cafile = None {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.449154 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] placement.certfile = None {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.450038 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] command line args: [] {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Aug 27 02:21:40.450588 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] placement.collect_timing = False {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.450588 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] placement.connect_retries = None {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.451246 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] config files: ['/etc/nova/api-paste.ini', '/etc/nova/nova.conf'] {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Aug 27 02:21:40.451510 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] placement.connect_retry_delay = None {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.451758 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] ================================================================================ {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Aug 27 02:21:40.452018 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] placement.endpoint_override = None {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.452018 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] placement.insecure = False {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.452587 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] allow_resize_to_same_host = True {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:21:40.452846 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] placement.keyfile = None {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.452846 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] placement.max_version = None {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.453372 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] arq_binding_timeout = 300 {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:21:40.453636 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] placement.min_version = None {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.453911 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] block_device_allocate_retries = 60 {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:21:40.454175 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] placement.region_name = RegionOne {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.454175 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] placement.service_name = None {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.454812 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] block_device_allocate_retries_interval = 3 {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:21:40.455074 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] placement.service_type = placement {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.455074 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] placement.split_loggers = False {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.455725 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] cert = self.pem {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:21:40.456808 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] placement.status_code_retries = None {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.457096 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] compute_driver = None {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:21:40.457378 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] placement.status_code_retry_delay = None {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.457378 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] placement.timeout = None {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.458060 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] compute_monitors = [] {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:21:40.458396 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] placement.valid_interfaces = ['internal', 'public'] {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.458396 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] placement.version = None {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.458996 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] config_dir = [] {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:21:40.459219 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] quota.cores = 20 {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.459219 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] quota.count_usage_from_placement = False {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.459776 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] config_drive_format = iso9660 {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:21:40.460041 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] quota.driver = nova.quota.DbQuotaDriver {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.460259 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] config_file = ['/etc/nova/api-paste.ini', '/etc/nova/nova.conf'] {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:21:40.460469 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_file_content_bytes = 10240 {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.460469 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_file_path_length = 255 {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.460876 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] config_source = [] {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:21:40.461291 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_files = 5 {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.461501 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] console_host = dv-cinder-846303-3-iscsi {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:21:40.461730 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] quota.instances = 10 {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.461730 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] quota.key_pairs = 100 {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.462189 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] control_exchange = nova {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:21:40.462408 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] quota.metadata_items = 128 {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.462408 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] quota.ram = 51200 {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.462861 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] cpu_allocation_ratio = None {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:21:40.463082 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] quota.recheck_quota = True {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.463290 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] daemon = False {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:21:40.463511 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] quota.server_group_members = 10 {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.463735 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] debug = True {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:21:40.463941 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] quota.server_groups = 10 {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.464156 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] default_access_ip_network_name = None {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:21:40.464361 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] rdp.enabled = False {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.464575 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] default_availability_zone = nova {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:21:40.464575 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] default_ephemeral_format = None {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:21:40.465208 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] rdp.html5_proxy_base_url = http://127.0.0.1:6083/ {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.465419 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'glanceclient=WARN', 'oslo.privsep.daemon=INFO'] {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:21:40.465688 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.discover_hosts_in_cells_interval = -1 {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.466031 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] default_schedule_zone = None {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:21:40.466270 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.enable_isolated_aggregate_filtering = False {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.466270 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.image_metadata_prefilter = False {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.467107 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] disk_allocation_ratio = None {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:21:40.467326 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.limit_tenants_to_placement_aggregate = False {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.467531 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] enable_new_services = True {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:21:40.467746 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.max_attempts = 3 {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.467977 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] enabled_apis = ['osapi_compute'] {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:21:40.468326 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.max_placement_results = 1000 {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.468541 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] enabled_ssl_apis = [] {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:21:40.468767 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.placement_aggregate_required_for_tenants = False {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.469170 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] flat_injected = False {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:21:40.469375 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_availability_zone = True {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.469611 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] force_config_drive = True {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:21:40.469960 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_image_type_support = False {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.470184 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] force_raw_images = True {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:21:40.470388 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_routed_network_aggregates = False {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.470595 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] graceful_shutdown_timeout = 5 {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:21:40.470801 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.workers = 2 {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.471025 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] heal_instance_info_cache_interval = 60 {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:21:40.471257 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.aggregate_image_properties_isolation_namespace = None {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.471257 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.aggregate_image_properties_isolation_separator = . {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.471820 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] host = dv-cinder-846303-3-iscsi {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:21:40.472024 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.available_filters = ['nova.scheduler.filters.all_filters'] {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.472256 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] initial_cpu_allocation_ratio = 16.0 {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:21:40.472473 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.build_failure_weight_multiplier = 1000000.0 {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.472473 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.cpu_weight_multiplier = 1.0 {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.472955 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] initial_disk_allocation_ratio = 1.0 {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:21:40.473211 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.cross_cell_move_weight_multiplier = 1000000.0 {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.473436 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] initial_ram_allocation_ratio = 1.5 {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:21:40.473642 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.disk_weight_multiplier = 1.0 {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.473642 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.enabled_filters = ['AvailabilityZoneFilter', 'ComputeFilter', 'ComputeCapabilitiesFilter', 'ImagePropertiesFilter', 'ServerGroupAntiAffinityFilter', 'ServerGroupAffinityFilter', 'SameHostFilter', 'DifferentHostFilter'] {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.474053 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] injected_network_template = /opt/stack/nova/nova/virt/interfaces.template {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:21:40.474272 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.host_subset_size = 1 {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.474476 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] instance_build_timeout = 0 {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:21:40.474802 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.image_properties_default_architecture = None {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.475029 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] instance_delete_interval = 300 {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:21:40.475408 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.io_ops_weight_multiplier = -1.0 {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.475408 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.isolated_hosts = [] {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.475822 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] instance_format = [instance: %(uuid)s] {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:21:40.477675 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.isolated_images = [] {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.477904 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] instance_name_template = instance-%08x {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:21:40.478121 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.max_instances_per_host = 50 {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.478520 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] instance_usage_audit = False {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:21:40.478761 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.max_io_ops_per_host = 8 {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.478761 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.pci_weight_multiplier = 1.0 {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.479417 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] instance_usage_audit_period = month {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:21:40.479632 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.ram_weight_multiplier = 1.0 {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.479843 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:21:40.480260 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.restrict_isolated_hosts_to_isolated_images = True {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.480483 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] instances_path = /opt/stack/data/nova/instances {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:21:40.480701 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.shuffle_best_same_weighed_hosts = False {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.480701 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.soft_affinity_weight_multiplier = 1.0 {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.481102 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] internal_service_availability_zone = internal {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:21:40.481371 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.soft_anti_affinity_weight_multiplier = 1.0 {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.481624 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] key = None {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:21:40.482008 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.track_instance_changes = False {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.482215 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] live_migration_retry_count = 30 {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:21:40.482419 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.weight_classes = ['nova.scheduler.weights.all_weighers'] {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.482640 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] log_config_append = None {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:21:40.482909 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] metrics.required = True {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.483139 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:21:40.483355 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_multiplier = 1.0 {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.483355 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_of_unavailable = -10000.0 {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.483773 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] log_dir = None {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:21:40.483773 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] log_file = None {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:21:40.484181 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_setting = [] {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.484398 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] log_options = True {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:21:40.484601 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.base_url = ws://127.0.0.1:6083/ {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.484808 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] log_rotate_interval = 1 {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:21:40.485022 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.enabled = False {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.485267 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] log_rotate_interval_type = days {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:21:40.485668 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.port_range = 10000:20000 {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.485892 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] log_rotation_type = none {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:21:40.486108 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.proxyclient_address = 127.0.0.1 {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.486108 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.serialproxy_host = 0.0.0.0 {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.486497 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:21:40.486726 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.serialproxy_port = 6083 {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.487282 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:21:40.487779 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] service_user.auth_section = None {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.487998 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:21:40.488207 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] service_user.auth_type = password {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.488436 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:21:40.488804 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] service_user.cafile = None {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.488804 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] service_user.certfile = None {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.489236 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:21:40.489657 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] service_user.collect_timing = False {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.489865 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] long_rpc_timeout = 1800 {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:21:40.490106 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] service_user.insecure = False {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.490459 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_builds = 10 {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:21:40.490741 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] service_user.keyfile = None {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.490972 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_live_migrations = 1 {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:21:40.491177 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] service_user.send_service_user_token = True {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.491384 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_snapshots = 5 {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:21:40.491384 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] max_local_block_devices = 3 {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:21:40.491793 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] service_user.split_loggers = False {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.492244 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] max_logfile_count = 30 {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:21:40.492454 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] service_user.timeout = None {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.492660 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] max_logfile_size_mb = 200 {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:21:40.492939 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] spice.agent_enabled = True {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.493233 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] maximum_instance_delete_attempts = 5 {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:21:40.493404 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] spice.enabled = False {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.493597 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] metadata_listen = 0.0.0.0 {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:21:40.494127 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_base_url = http://127.0.0.1:6082/spice_auto.html {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.494369 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] metadata_listen_port = 8775 {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:21:40.494369 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] metadata_workers = 2 {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:21:40.494753 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_host = 0.0.0.0 {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.494923 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] migrate_max_retries = -1 {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:21:40.495056 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_port = 6082 {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.495197 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] mkisofs_cmd = genisoimage {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:21:40.495325 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] spice.server_listen = 127.0.0.1 {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.495460 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] my_block_storage_ip = 10.222.0.52 {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:21:40.495460 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] my_ip = 10.222.0.52 {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:21:40.495748 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] spice.server_proxyclient_address = 127.0.0.1 {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.495883 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] network_allocate_retries = 0 {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:21:40.496173 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.baseapi = None {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.496289 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] non_inheritable_image_properties = ['cache_in_nova', 'bittorrent'] {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:21:40.496425 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.cert = None {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.496562 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_listen = 0.0.0.0 {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:21:40.496747 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.compute = auto {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.496941 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_listen_port = 8774 {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:21:40.497049 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.conductor = None {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.497282 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_unique_server_name_scope = {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:21:40.497434 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.scheduler = None {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.497548 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_workers = 2 {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:21:40.497548 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] password_length = 12 {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:21:40.497941 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.auth_section = None {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.498029 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] periodic_enable = True {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:21:40.498112 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.auth_type = None {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.498209 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] periodic_fuzzy_delay = 60 {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:21:40.498209 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] pointer_model = usbtablet {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:21:40.498361 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.cafile = None {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.498443 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] preallocate_images = none {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:21:40.498524 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.certfile = None {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.498614 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] publish_errors = False {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:21:40.498614 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] pybasedir = /opt/stack/nova {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:21:40.498859 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.collect_timing = False {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.498958 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] ram_allocation_ratio = None {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:21:40.498958 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_burst = 0 {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:21:40.499119 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.insecure = False {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.499205 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_except_level = CRITICAL {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:21:40.499205 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_interval = 0 {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:21:40.499368 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.keyfile = None {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.499459 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] reboot_timeout = 0 {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:21:40.499459 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] reclaim_instance_interval = 0 {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:21:40.499639 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.split_loggers = False {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.499733 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] record = None {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:21:40.499733 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] report_interval = 10 {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:21:40.499885 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.timeout = None {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.499968 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] rescue_timeout = 0 {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:21:40.499968 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_cpus = 0 {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:21:40.500127 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] vmware.api_retry_count = 10 {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.500219 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_disk_mb = 0 {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:21:40.500219 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_memory_mb = 512 {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:21:40.500622 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] vmware.ca_file = None {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.500716 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] reserved_huge_pages = None {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:21:40.500716 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] resize_confirm_window = 0 {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:21:40.500716 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] resize_fs_using_block_device = False {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:21:40.500958 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] vmware.cache_prefix = None {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.501043 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] resume_guests_state_on_host_boot = False {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:21:40.501163 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] vmware.cluster_name = None {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.501268 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] rootwrap_config = /etc/nova/rootwrap.conf {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:21:40.501268 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_action = reap {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:21:40.501503 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] vmware.connection_pool_size = 10 {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.501609 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_poll_interval = 1800 {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:21:40.501703 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_timeout = 0 {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:21:40.501855 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] vmware.console_delay_seconds = None {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.501989 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] scheduler_instance_sync_interval = 120 {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:21:40.502269 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] service_down_time = 60 {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:21:40.502454 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] vmware.datastore_regex = None {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.502612 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] servicegroup_driver = db {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:21:40.502749 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_ip = None {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.502880 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] shelved_offload_time = 0 {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:21:40.503022 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_password = **** {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.503190 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] shelved_poll_interval = 3600 {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:21:40.503350 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_port = 443 {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.503519 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] shutdown_timeout = 0 {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:21:40.503833 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] source_is_ipv6 = False {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:21:40.503974 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_username = None {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.504143 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] ssl_only = False {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:21:40.504391 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] vmware.insecure = False {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.504495 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] state_path = /opt/stack/data/nova {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:21:40.504767 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] sync_power_state_interval = 600 {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:21:40.505064 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] vmware.integration_bridge = None {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.505322 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] sync_power_state_pool_size = 1000 {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:21:40.505479 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] syslog_log_facility = LOG_USER {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:21:40.505618 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] vmware.maximum_objects = 100 {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.505769 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] tempdir = None {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:21:40.506015 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_default_policy = None {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.506158 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] timeout_nbd = 10 {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:21:40.506383 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] transport_url = **** {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:21:40.506598 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_enabled = False {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.506697 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] update_resources_interval = 0 {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:21:40.506932 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] use_cow_images = True {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:21:40.507122 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_wsdl_location = None {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.507263 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] use_eventlog = False {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:21:40.507533 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] use_journal = False {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:21:40.507632 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_log_dir = /opt/vmware/vspc {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.507847 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] use_json = False {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:21:40.508016 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_port_proxy_uri = None {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.508146 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] use_rootwrap_daemon = False {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:21:40.508413 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_port_service_uri = None {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.508567 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] use_stderr = False {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:21:40.508706 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] vmware.task_poll_interval = 0.5 {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.508965 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] use_syslog = False {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:21:40.509104 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] vcpu_pin_set = None {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:21:40.509335 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] vmware.use_linked_clone = True {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.509493 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] vif_plugging_is_fatal = True {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:21:40.509706 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_keymap = en-us {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.509820 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] vif_plugging_timeout = 300 {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:21:40.510040 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_port = 5900 {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.510166 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] virt_mkfs = [] {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:21:40.510349 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] volume_usage_poll_interval = 0 {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:21:40.510455 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_port_total = 10000 {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.510714 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] watch_log_file = False {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:21:40.510871 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] vnc.auth_schemes = ['none'] {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.511003 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] web = /usr/share/spice-html5 {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 27 02:21:40.511097 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] vnc.enabled = True {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.511310 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] oslo_concurrency.disable_process_locking = False {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.511650 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] oslo_concurrency.lock_path = /opt/stack/data/nova {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.511753 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_base_url = http://127.0.0.1:6080/vnc_auto.html {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.511994 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] api.auth_strategy = keystone {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.512127 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_host = 0.0.0.0 {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.512312 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] api.compute_link_prefix = None {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.512598 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_port = 6080 {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.512796 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] api.config_drive_skip_versions = 1.0 2007-01-19 2007-03-01 2007-08-29 2007-10-10 2007-12-15 2008-02-01 2008-09-01 {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.512957 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] api.dhcp_domain = novalocal {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.513138 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] vnc.server_listen = 127.0.0.1 {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.513319 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] api.enable_instance_password = True {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.513512 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] vnc.server_proxyclient_address = 127.0.0.1 {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.513766 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] api.glance_link_prefix = None {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.513936 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_ca_certs = None {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.514034 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_cells_batch_fixed_size = 100 {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.514119 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_client_cert = None {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.514253 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_cells_batch_strategy = distributed {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.514358 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_client_key = None {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.514568 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_per_project_cells = False {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.514719 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_compute_service_check_for_ffu = False {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.514883 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] api.list_records_by_skipping_down_cells = True {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.515141 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_fallback_pcpu_query = False {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.515238 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] api.local_metadata_per_cell = False {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.515505 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] api.max_limit = 1000 {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.515600 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_group_policy_check_upcall = False {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.515826 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] api.metadata_cache_expiration = 15 {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.516010 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_libvirt_livesnapshot = False {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.516149 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] api.neutron_default_tenant_id = default {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.516444 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_rootwrap = False {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.516557 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] api.use_forwarded_for = False {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.516775 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] api.use_neutron_default_nets = False {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.516873 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.enable_numa_live_migration = False {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.517101 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_connect_timeout = 5 {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.517393 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.enable_qemu_monitor_announce_self = False {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.517503 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_failure_fatal = False {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.517813 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_read_timeout = 5 {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.518031 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.ensure_libvirt_rbd_instance_dir_cleanup = False {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.518214 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_ssl_certfile = {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.518381 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.handle_virt_lifecycle_events = True {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.518490 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_targets = [] {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.518583 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.libvirt_disable_apic = False {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.518715 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_jsonfile_path = None {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.518960 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.never_download_image_if_on_rbd = False {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.519119 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_providers = ['StaticJSON'] {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.519402 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] cache.backend = dogpile.cache.memcached {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.519559 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.reserve_disk_resource_for_image_cache = False {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.519763 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] cache.backend_argument = **** {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.519960 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.skip_cpu_compare_on_dest = False {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.520882 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] cache.config_prefix = cache.oslo {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.521337 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.skip_hypervisor_version_check_on_lm = False {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.521747 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] cache.dead_timeout = 60.0 {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.521747 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] cache.debug_cache_backend = False {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.522594 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.unified_limits_count_pcpu_as_vcpu = False {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.523059 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] cache.enable_retry_client = False {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.523059 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] cache.enable_socket_keepalive = False {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.523834 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.wait_for_vif_plugged_event_during_hard_reboot = [] {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.524147 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] cache.enabled = True {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.524411 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.api_paste_config = /etc/nova/api-paste.ini {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.524892 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] cache.expiration_time = 600 {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.525215 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.client_socket_timeout = 900 {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.525488 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] cache.hashclient_retry_attempts = 2 {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.525488 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] cache.hashclient_retry_delay = 1.0 {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.525488 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_dead_retry = 300 {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.526144 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.default_pool_size = 1000 {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.526524 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_password = {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.526524 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.526524 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.526524 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_maxsize = 10 {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.527401 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.keep_alive = True {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.527685 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_unused_timeout = 60 {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.528121 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.max_header_line = 16384 {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.528121 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.secure_proxy_ssl_header = None {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.528121 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_ca_file = None {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.528121 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_cert_file = None {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.528121 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_key_file = None {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.528121 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.tcp_keepidle = 600 {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.528121 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.wsgi_log_format = %(client_ip)s "%(request_line)s" status: %(status_code)s len: %(body_length)s time: %(wall_seconds).7f {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.528121 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] zvm.ca_file = None {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.529148 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] zvm.cloud_connector_url = None {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.529148 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] zvm.image_tmp_path = /opt/stack/data/nova/images {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.529584 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] zvm.reachable_timeout = 300 {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.530055 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] oslo_versionedobjects.fatal_exception_format_errors = False {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.530548 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.530644 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_sasl_enabled = False {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.531002 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_servers = ['localhost:11211'] {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.531153 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.531446 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_socket_timeout = 1.0 {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.531579 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_process_name = {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.531805 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_username = {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.532093 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.532268 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] cache.proxies = [] {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.532480 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] cache.retry_attempts = 2 {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.532589 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.532819 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] cache.retry_delay = 0.0 {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.533051 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.enforce_new_defaults = False {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.533249 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_count = 1 {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.533692 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_idle = 1 {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.534057 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.enforce_scope = False {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.534387 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_interval = 1 {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.534625 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_default_rule = default {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.534721 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_allowed_ciphers = None {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.534721 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_cafile = None {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.534922 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.535074 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_certfile = None {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.535371 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_enabled = False {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.535470 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_file = policy.yaml {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.535690 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_keyfile = None {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.535952 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.536044 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] cinder.auth_section = None {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.536565 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] cinder.auth_type = password {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.537002 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.537109 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] cinder.cafile = None {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.537109 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] cinder.catalog_info = volumev3::publicURL {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.537339 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.537425 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] cinder.certfile = None {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.537660 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.537758 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] cinder.collect_timing = False {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.538000 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] cinder.cross_az_attach = True {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.538203 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.538326 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] cinder.debug = False {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.538617 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] cinder.endpoint_template = None {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.538805 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.auth_section = None {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.538967 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] cinder.http_retries = 3 {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.539251 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] cinder.insecure = False {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.539407 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.auth_type = None {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.539545 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] cinder.keyfile = None {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.539745 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.cafile = None {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.539911 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] cinder.os_region_name = RegionOne {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.540325 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.certfile = None {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.542853 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.collect_timing = False {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.543025 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] cinder.split_loggers = False {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.543025 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] cinder.timeout = None {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.543286 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.connect_retries = None {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.543446 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] compute.consecutive_build_service_disable_threshold = 10 {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.543591 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.connect_retry_delay = None {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.543746 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] compute.cpu_dedicated_set = None {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.543879 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.endpoint_id = None {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.544021 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] compute.cpu_shared_set = None {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.544164 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.endpoint_override = None {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.544304 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] compute.image_type_exclude_list = [] {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.544429 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.insecure = False {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.544562 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] compute.live_migration_wait_for_vif_plug = True {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.544698 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.keyfile = None {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.544837 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] compute.max_concurrent_disk_ops = 0 {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.544984 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.max_version = None {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.545112 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] compute.max_disk_devices_to_attach = -1 {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.545274 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.min_version = None {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.545409 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] compute.packing_host_numa_cells_allocation_strategy = False {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.545536 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.region_name = None {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.545684 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] compute.provider_config_location = /etc/nova/provider_config/ {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.545848 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.service_name = None {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.545998 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] compute.resource_provider_association_refresh = 300 {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.546138 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.service_type = None {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.546275 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] compute.shutdown_retry_interval = 10 {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.546410 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.split_loggers = False {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.546549 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] conductor.workers = 2 {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.546679 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.status_code_retries = None {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.546679 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.status_code_retry_delay = None {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.546917 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] console.allowed_origins = [] {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.547061 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.timeout = None {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.547204 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] console.ssl_ciphers = None {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.547364 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.valid_interfaces = None {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.547521 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] console.ssl_minimum_version = default {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.547666 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.version = None {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.547808 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] consoleauth.token_ttl = 600 {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.547808 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.cafile = None {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.548092 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.548277 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.certfile = None {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.548455 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] profiler.connection_string = messaging:// {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.548612 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.collect_timing = False {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.548762 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] profiler.enabled = False {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.548905 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.connect_retries = None {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.549062 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_doc_type = notification {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.549233 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.connect_retry_delay = None {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.549233 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.endpoint_override = None {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.549506 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_scroll_size = 10000 {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.549643 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.insecure = False {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.549811 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_scroll_time = 2m {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.549940 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.keyfile = None {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.550073 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] profiler.filter_error_trace = False {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.550291 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.max_version = None {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.550420 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] profiler.hmac_keys = SECRET_KEY {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.550546 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.min_version = None {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.550682 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] profiler.sentinel_service_name = mymaster {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.550820 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.region_name = None {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.550955 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] profiler.socket_timeout = 0.1 {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.551075 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.service_name = None {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.551198 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] profiler.trace_sqlalchemy = False {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.551326 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.service_type = accelerator {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.551455 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.amqp_auto_delete = False {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.551577 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.split_loggers = False {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.551697 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.amqp_durable_queues = False {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.551820 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.status_code_retries = None {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.551951 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.conn_pool_min_size = 2 {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.552189 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.status_code_retry_delay = None {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.552336 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.conn_pool_ttl = 1200 {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.552479 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.timeout = None {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.552608 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.direct_mandatory_flag = True {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.552739 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.valid_interfaces = ['internal', 'public'] {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.552865 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.enable_cancel_on_failover = False {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.552996 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.version = None {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.553182 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_in_pthread = False {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.553319 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] database.backend = sqlalchemy {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.553468 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_rate = 3 {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.553871 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] database.connection = **** {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.554119 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_timeout_threshold = 600 {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.554348 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_debug = 0 {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.554542 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_compression = None {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.554734 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_parameters = {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.554933 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_failover_strategy = round-robin {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.555134 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_recycle_time = 3600 {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.555339 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_missing_consumer_retry_timeout = 60 {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.555517 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_trace = False {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.555752 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_reconnect_delay = 1.0 {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.555934 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] database.db_inc_retry_interval = True {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.556125 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_ha_queues = False {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.556472 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] database.db_max_retries = 20 {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.556690 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_interval_max = 30 {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.556883 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] database.db_max_retry_interval = 10 {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.557081 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_login_method = AMQPLAIN {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.557377 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] database.db_retry_interval = 1 {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.557560 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_qos_prefetch_count = 0 {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.557737 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] database.max_overflow = 50 {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.557737 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] database.max_pool_size = 5 {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.558079 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quorum_delivery_limit = 0 {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.558244 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] database.max_retries = 10 {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.558466 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quorum_queue = False {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.558684 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] database.mysql_enable_ndb = False {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.558684 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] database.mysql_sql_mode = TRADITIONAL {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.559100 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quroum_max_memory_bytes = 0 {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.559304 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] database.pool_timeout = None {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.559503 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quroum_max_memory_length = 0 {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.559714 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] database.retry_interval = 10 {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.559912 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_retry_backoff = 2 {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.559912 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_retry_interval = 1 {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.559912 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_transient_queues_ttl = 1800 {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.560512 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] database.slave_connection = **** {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.560732 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rpc_conn_pool_size = 30 {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.560962 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] database.sqlite_synchronous = True {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.561219 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl = False {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.561498 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] api_database.backend = sqlalchemy {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.561706 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_ca_file = {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.561909 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection = **** {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.562134 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_cert_file = {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.562338 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_debug = 0 {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.562528 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_enforce_fips_mode = False {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.562753 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_parameters = {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.562963 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_key_file = {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.563166 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_recycle_time = 3600 {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.563368 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_version = {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.563574 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_trace = False {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.563788 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.driver = ['messagingv2'] {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.563982 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_inc_retry_interval = True {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.564181 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.retry = -1 {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.564413 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_max_retries = 20 {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.564413 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_max_retry_interval = 10 {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.564413 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_retry_interval = 1 {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.564413 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_overflow = 50 {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.565397 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.topics = ['notifications'] {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.565596 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_pool_size = 5 {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.565596 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_retries = 10 {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.565918 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.transport_url = **** {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.566083 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] api_database.mysql_enable_ndb = False {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.566255 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.file_event_handler = None {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.566423 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] api_database.mysql_sql_mode = TRADITIONAL {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.566539 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.file_event_handler_interval = 1 {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.566656 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] api_database.pool_timeout = None {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.566779 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.log_dir = None {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.566779 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi_app [-] ******************************************************************************** {{(pid=120863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Aug 27 02:21:40.567084 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] api_database.retry_interval = 10 {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.567084 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] api_database.slave_connection = **** {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.567084 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] api_database.sqlite_synchronous = True {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.567084 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] devices.enabled_mdev_types = [] {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.567084 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.cipher = aes-xts-plain64 {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.567084 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.enabled = False {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.567084 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.key_size = 512 {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.567084 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] glance.api_servers = None {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.568340 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] glance.cafile = None {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.568340 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] glance.certfile = None {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.568340 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] glance.collect_timing = False {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.568340 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] glance.connect_retries = None {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.568340 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] glance.connect_retry_delay = None {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.568340 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] glance.debug = False {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.569091 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] glance.default_trusted_certificate_ids = [] {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.569091 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] glance.enable_certificate_validation = False {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.569366 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] glance.enable_rbd_download = False {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.569574 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] glance.endpoint_override = None {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.569892 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] glance.insecure = False {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.570204 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] glance.keyfile = None {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.570563 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] glance.max_version = None {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.570879 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] glance.min_version = None {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.571201 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] glance.num_retries = 3 {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.571620 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_ceph_conf = {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.573362 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_connect_timeout = 5 {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.573362 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_pool = {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.573362 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_user = {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.573362 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] glance.region_name = None {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.573362 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] glance.service_name = None {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.574290 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] glance.service_type = image {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.574290 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] glance.split_loggers = False {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.574581 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] glance.status_code_retries = None {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.574792 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] glance.status_code_retry_delay = None {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.575395 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] glance.timeout = None {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.575762 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] glance.valid_interfaces = ['internal', 'public'] {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.576085 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] glance.verify_glance_signatures = False {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.576396 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] glance.version = None {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.576720 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] guestfs.debug = False {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.577148 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.config_drive_cdrom = False {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.577468 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.config_drive_inject_password = False {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.577939 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.dynamic_memory_ratio = 1.0 {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.578145 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.enable_instance_metrics_collection = False {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.578463 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.enable_remotefx = False {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.578796 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.instances_path_share = {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.579115 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.iscsi_initiator_list = [] {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.579437 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.limit_cpu_features = False {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.579752 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.mounted_disk_query_retry_count = 10 {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.580071 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.mounted_disk_query_retry_interval = 5 {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.580382 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.power_state_check_timeframe = 60 {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.580699 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.power_state_event_polling_interval = 2 {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.581024 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.qemu_img_cmd = qemu-img.exe {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.581359 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.use_multipath_io = False {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.581673 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.volume_attach_retry_count = 10 {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.581986 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.volume_attach_retry_interval = 5 {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.582294 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.vswitch_name = None {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.582607 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.wait_soft_reboot_seconds = 60 {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.582995 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] mks.enabled = False {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.583747 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] mks.mksproxy_base_url = http://127.0.0.1:6090/ {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.584084 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.manager_interval = 2400 {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.584403 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.precache_concurrency = 1 {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.584729 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_base_images = True {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.585056 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_original_minimum_age_seconds = 86400 {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.585435 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_resized_minimum_age_seconds = 3600 {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.585907 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.subdirectory_name = _base {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.586138 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] ironic.api_max_retries = 60 {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.586407 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] ironic.api_retry_interval = 2 {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.586718 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] ironic.auth_section = None {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.587333 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] ironic.auth_type = None {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.589412 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] ironic.cafile = None {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.589751 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] ironic.certfile = None {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.590124 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] ironic.collect_timing = False {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.590844 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] ironic.connect_retries = None {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.591173 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] ironic.connect_retry_delay = None {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.591496 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] ironic.endpoint_override = None {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.591958 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] ironic.insecure = False {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.592282 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] ironic.keyfile = None {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.593613 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] ironic.max_version = None {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.593947 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] ironic.min_version = None {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.594284 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] ironic.partition_key = None {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.594627 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] ironic.peer_list = [] {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.594954 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] ironic.region_name = None {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.595288 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] ironic.serial_console_state_timeout = 10 {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.595616 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] ironic.service_name = None {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.596129 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] ironic.service_type = baremetal {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.596300 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] ironic.split_loggers = False {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.596580 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] ironic.status_code_retries = None {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.596897 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] ironic.status_code_retry_delay = None {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.597237 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] ironic.timeout = None {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.597658 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] ironic.valid_interfaces = ['internal', 'public'] {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.598036 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] ironic.version = None {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.598415 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] key_manager.backend = nova.keymgr.conf_key_mgr.ConfKeyManager {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.598646 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] key_manager.fixed_key = **** {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.598981 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] barbican.auth_endpoint = http://localhost/identity/v3 {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.599294 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_api_version = None {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.599610 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_endpoint = None {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.599943 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_endpoint_type = public {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.600250 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_region_name = None {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.600559 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] barbican.cafile = None {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.600867 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] barbican.certfile = None {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.601207 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] barbican.collect_timing = False {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.601526 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] barbican.insecure = False {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.601840 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] barbican.keyfile = None {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.602131 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] barbican.number_of_retries = 60 {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.602444 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] barbican.retry_delay = 1 {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.602793 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] barbican.send_service_user_token = False {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.603260 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] barbican.split_loggers = False {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.603798 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] barbican.timeout = None {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.603987 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] barbican.verify_ssl = True {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.604180 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] barbican.verify_ssl_path = None {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.604502 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.auth_section = None {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.604816 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.auth_type = None {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.605160 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.cafile = None {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.605452 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.certfile = None {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.605768 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.collect_timing = False {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.606232 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.insecure = False {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.606411 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.keyfile = None {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.606704 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.split_loggers = False {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.607013 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.timeout = None {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.607333 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] vault.approle_role_id = None {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.607642 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] vault.approle_secret_id = None {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.607949 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] vault.cafile = None {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.608256 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] vault.certfile = None {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.608567 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] vault.collect_timing = False {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.608878 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] vault.insecure = False {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.609202 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] vault.keyfile = None {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.609536 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] vault.kv_mountpoint = secret {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.609932 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] vault.kv_version = 2 {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.610186 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] vault.namespace = None {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.610568 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] vault.root_token_id = None {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.610846 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] vault.split_loggers = False {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.611157 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] vault.ssl_ca_crt_file = None {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.611463 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] vault.timeout = None {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.611776 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] vault.use_ssl = False {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.612105 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] vault.vault_url = http://127.0.0.1:8200 {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.612427 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] keystone.cafile = None {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.612771 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] keystone.certfile = None {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.613057 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] keystone.collect_timing = False {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.613394 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] keystone.connect_retries = None {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.613712 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] keystone.connect_retry_delay = None {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.614020 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] keystone.endpoint_override = None {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.614331 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] keystone.insecure = False {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.614681 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] keystone.keyfile = None {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.614944 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] keystone.max_version = None {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.615249 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] keystone.min_version = None {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.615559 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] keystone.region_name = None {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.615864 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] keystone.service_name = None {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.616240 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] keystone.service_type = identity {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.616524 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] keystone.split_loggers = False {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.616791 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] keystone.status_code_retries = None {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.617093 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] keystone.status_code_retry_delay = None {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.617423 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] keystone.timeout = None {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.617782 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] keystone.valid_interfaces = ['internal', 'public'] {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.618373 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] keystone.version = None {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.618852 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.connection_uri = {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.619197 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_mode = None {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.619544 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_model_extra_flags = [] {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.619896 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_models = [] {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.620238 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.device_detach_attempts = 8 {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.620581 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.device_detach_timeout = 20 {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.620932 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.disk_cachemodes = [] {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.621483 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.disk_prefix = None {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.621900 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.enabled_perf_events = [] {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.622232 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.file_backed_memory = 0 {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.622562 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.gid_maps = [] {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.622881 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.hw_disk_discard = None {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.623197 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.hw_machine_type = None {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.623531 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_ceph_conf = {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.623853 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_copy_poll_interval = 15 {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.624325 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_copy_timeout = 600 {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.624872 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_store_name = {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.625303 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_pool = rbd {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.625863 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_type = default {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.626226 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_volume_group = None {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.626674 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_key = False {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.627151 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_partition = -2 {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.627820 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_password = False {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.628357 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.iscsi_iface = None {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.628735 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.iser_use_multipath = False {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.629065 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_bandwidth = 0 {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.629417 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_completion_timeout = 800 {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.629742 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime = 500 {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.630065 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime_delay = 75 {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.630981 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime_steps = 10 {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.630981 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_inbound_addr = None {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.631455 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_permit_auto_converge = False {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.632113 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_permit_post_copy = False {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.632514 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_scheme = None {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.633207 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_timeout_action = abort {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.633739 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_tunnelled = False {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.634122 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_uri = None {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.634681 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_with_native_tls = False {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.635227 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.max_queues = None {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.635805 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.mem_stats_period_seconds = 10 {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.636310 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.nfs_mount_options = None {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.637065 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.nfs_mount_point_base = /opt/stack/data/nova/mnt {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.637620 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_aoe_discover_tries = 3 {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.638145 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_iser_scan_tries = 5 {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.638662 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_memory_encrypted_guests = None {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.639224 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_nvme_discover_tries = 5 {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.640117 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_pcie_ports = 0 {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.640659 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_volume_scan_tries = 5 {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.641218 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.pmem_namespaces = [] {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.641793 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.quobyte_client_cfg = None {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.642625 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.quobyte_mount_point_base = /opt/stack/data/nova/mnt {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.644710 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO dbcounter [None req-0a2732d3-b585-470e-97cc-a0a3ddcec5ff None None] Registered counter for database nova_api Aug 27 02:21:40.647094 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_connect_timeout = 5 {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.647729 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_destroy_volume_retries = 12 {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.648325 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_destroy_volume_retry_interval = 5 {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.648901 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_secret_uuid = None {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.649618 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_user = None {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.650206 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.realtime_scheduler_priority = 1 {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.650829 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.remote_filesystem_transport = ssh {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.651428 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_image_id = None {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.651988 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_kernel_id = None {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.652554 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_ramdisk_id = None {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.653239 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rng_dev_path = /dev/urandom {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.653744 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rx_queue_size = None {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.654347 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.smbfs_mount_options = {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.655157 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.smbfs_mount_point_base = /opt/stack/data/nova/mnt {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.655749 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshot_compression = False {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.656211 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshot_image_format = None {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.656710 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshots_directory = /opt/stack/data/nova/instances/snapshots {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.656997 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.sparse_logical_volumes = False {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.657436 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_enabled = False {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.657673 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_group = tss {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.658004 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_user = tss {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.658374 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.sysinfo_serial = unique {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.659079 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.tx_queue_size = None {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.659079 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.uid_maps = [] {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.659590 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.use_virtio_for_bridges = True {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.659829 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.virt_type = kvm {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.660229 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_clear = zero {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.660578 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_clear_size = 0 {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.660826 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_use_multipath = False {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.661061 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_cache_path = None {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.661362 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_log_path = /var/log/vstorage/%(cluster_name)s/nova.log.gz {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.661712 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_group = qemu {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.662723 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_opts = [] {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.663689 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_perms = 0770 {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.664482 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_point_base = /opt/stack/data/nova/mnt {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.665081 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_user = stack {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.665777 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.wait_soft_reboot_seconds = 120 {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.666470 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] neutron.auth_section = None {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.667000 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] neutron.auth_type = password {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.667586 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] neutron.cafile = None {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.668198 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] neutron.certfile = None {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.668443 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_db.sqlalchemy.engines [None req-0a2732d3-b585-470e-97cc-a0a3ddcec5ff None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=120863) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Aug 27 02:21:40.668551 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] neutron.collect_timing = False {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.668801 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] neutron.connect_retries = None {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.669146 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] neutron.connect_retry_delay = None {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.669562 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] neutron.default_floating_pool = public {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.670133 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] neutron.endpoint_override = None {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.670655 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] neutron.extension_sync_interval = 600 {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.671227 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] neutron.http_retries = 3 {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.671792 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] neutron.insecure = False {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.672315 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] neutron.keyfile = None {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.672845 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] neutron.max_version = None {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.673435 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] neutron.metadata_proxy_shared_secret = **** {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.674122 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] neutron.min_version = None {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.674487 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] neutron.ovs_bridge = br-int {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.674916 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] neutron.physnets = [] {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.675461 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] neutron.region_name = RegionOne {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.676705 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_metadata_proxy = True {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.676705 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_name = None {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.677098 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_type = network {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.677624 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] neutron.split_loggers = False {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.678187 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] neutron.status_code_retries = None {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.678341 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG dbcounter [-] [120863] Writer thread running {{(pid=120863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Aug 27 02:21:40.678852 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] neutron.status_code_retry_delay = None {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.679302 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] neutron.timeout = None {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.679913 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] neutron.valid_interfaces = ['internal', 'public'] {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.680652 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] neutron.version = None {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.681218 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] notifications.bdms_in_notifications = False {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.682084 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] notifications.default_level = INFO {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.682590 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] notifications.notification_format = unversioned {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.683143 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] notifications.notify_on_state_change = None {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.683776 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] notifications.versioned_notifications_topics = ['versioned_notifications'] {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.684302 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] pci.alias = [] {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.684904 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] pci.device_spec = [] {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.685490 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] placement.auth_section = None {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.686084 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] placement.auth_type = password {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.686654 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] placement.cafile = None {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.687225 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] placement.certfile = None {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.687813 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] placement.collect_timing = False {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.688332 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] placement.connect_retries = None {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.688657 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] placement.connect_retry_delay = None {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.688975 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] placement.endpoint_override = None {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.689379 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] placement.insecure = False {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.689852 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] placement.keyfile = None {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.690390 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] placement.max_version = None {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.690921 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] placement.min_version = None {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.691273 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] placement.region_name = RegionOne {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.691655 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] placement.service_name = None {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.692052 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] placement.service_type = placement {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.692382 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] placement.split_loggers = False {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.692821 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] placement.status_code_retries = None {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.693314 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] placement.status_code_retry_delay = None {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.693815 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] placement.timeout = None {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.694397 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] placement.valid_interfaces = ['internal', 'public'] {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.694935 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] placement.version = None {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.695533 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] quota.cores = 20 {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.696108 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] quota.count_usage_from_placement = False {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.697000 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] quota.driver = nova.quota.DbQuotaDriver {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.697785 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_file_content_bytes = 10240 {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.698366 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_file_path_length = 255 {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.698950 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_files = 5 {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.699542 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] quota.instances = 10 {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.700407 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] quota.key_pairs = 100 {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.700951 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] quota.metadata_items = 128 {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.701515 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] quota.ram = 51200 {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.702059 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] quota.recheck_quota = True {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.702640 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] quota.server_group_members = 10 {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.703238 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] quota.server_groups = 10 {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.703835 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] rdp.enabled = False {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.704775 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] rdp.html5_proxy_base_url = http://127.0.0.1:6083/ {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.705360 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.discover_hosts_in_cells_interval = -1 {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.705917 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.enable_isolated_aggregate_filtering = False {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.706490 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.image_metadata_prefilter = False {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.707082 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.limit_tenants_to_placement_aggregate = False {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.707677 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.max_attempts = 3 {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.708276 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.max_placement_results = 1000 {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.708851 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.placement_aggregate_required_for_tenants = False {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.709471 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_availability_zone = True {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.715704 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_image_type_support = False {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.715704 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_routed_network_aggregates = False {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.715704 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.workers = 2 {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.715704 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.aggregate_image_properties_isolation_namespace = None {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.715704 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.aggregate_image_properties_isolation_separator = . {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.715704 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.available_filters = ['nova.scheduler.filters.all_filters'] {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.715704 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.build_failure_weight_multiplier = 1000000.0 {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.715704 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.cpu_weight_multiplier = 1.0 {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.716353 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.cross_cell_move_weight_multiplier = 1000000.0 {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.716353 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.disk_weight_multiplier = 1.0 {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.716353 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.enabled_filters = ['AvailabilityZoneFilter', 'ComputeFilter', 'ComputeCapabilitiesFilter', 'ImagePropertiesFilter', 'ServerGroupAntiAffinityFilter', 'ServerGroupAffinityFilter', 'SameHostFilter', 'DifferentHostFilter'] {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.717183 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.host_subset_size = 1 {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.717544 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.image_properties_default_architecture = None {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.717933 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.io_ops_weight_multiplier = -1.0 {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.718495 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.isolated_hosts = [] {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.719041 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.isolated_images = [] {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.719605 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.max_instances_per_host = 50 {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.720178 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.max_io_ops_per_host = 8 {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.720776 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.pci_weight_multiplier = 1.0 {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.721682 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.ram_weight_multiplier = 1.0 {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.721967 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.restrict_isolated_hosts_to_isolated_images = True {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.722561 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.shuffle_best_same_weighed_hosts = False {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.723141 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.soft_affinity_weight_multiplier = 1.0 {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.723683 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.soft_anti_affinity_weight_multiplier = 1.0 {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.724237 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.track_instance_changes = False {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.724798 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.weight_classes = ['nova.scheduler.weights.all_weighers'] {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.725372 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] metrics.required = True {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.725913 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_multiplier = 1.0 {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.726465 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_of_unavailable = -10000.0 {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.727020 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_setting = [] {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.728016 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.base_url = ws://127.0.0.1:6083/ {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.728581 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.enabled = False {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.729215 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.port_range = 10000:20000 {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.729793 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.proxyclient_address = 127.0.0.1 {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.730334 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.serialproxy_host = 0.0.0.0 {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.730870 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.serialproxy_port = 6083 {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.731438 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] service_user.auth_section = None {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.732017 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] service_user.auth_type = password {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.732588 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] service_user.cafile = None {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.733171 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] service_user.certfile = None {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.733688 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] service_user.collect_timing = False {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.734213 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] service_user.insecure = False {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.734750 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] service_user.keyfile = None {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.735335 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] service_user.send_service_user_token = True {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.735904 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] service_user.split_loggers = False {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.736444 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] service_user.timeout = None {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.737039 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] spice.agent_enabled = True {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.737653 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] spice.enabled = False {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.738621 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_base_url = http://127.0.0.1:6082/spice_auto.html {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.739239 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_host = 0.0.0.0 {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.739826 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_port = 6082 {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.740392 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] spice.server_listen = 127.0.0.1 {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.741010 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] spice.server_proxyclient_address = 127.0.0.1 {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.741556 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.baseapi = None {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.742140 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.cert = None {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.742737 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.compute = auto {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.743385 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.conductor = None {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.743945 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.scheduler = None {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.744483 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.auth_section = None {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.745007 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.auth_type = None {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.745586 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.cafile = None {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.746148 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.certfile = None {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.746743 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.collect_timing = False {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.747313 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.insecure = False {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.747863 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.keyfile = None {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.748411 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.split_loggers = False {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.748953 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.timeout = None {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.749544 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] vmware.api_retry_count = 10 {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.750091 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] vmware.ca_file = None {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.750632 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] vmware.cache_prefix = None {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.751205 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] vmware.cluster_name = None {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.751788 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] vmware.connection_pool_size = 10 {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.752354 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] vmware.console_delay_seconds = None {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.752919 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] vmware.datastore_regex = None {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.753536 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_ip = None {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.754114 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_password = **** {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.754697 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_port = 443 {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.755268 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_username = None {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.755843 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] vmware.insecure = False {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.756366 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] vmware.integration_bridge = None {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.756906 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] vmware.maximum_objects = 100 {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.757481 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_default_policy = None {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.758023 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_enabled = False {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.758631 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_wsdl_location = None {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.759224 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_log_dir = /opt/vmware/vspc {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.759830 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_port_proxy_uri = None {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.760273 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.context [None req-0a2732d3-b585-470e-97cc-a0a3ddcec5ff None None] Found 2 cells: 00000000-0000-0000-0000-000000000000(cell0),41dc7c6b-010c-43dc-bd7e-0984fcde0c35(cell1) {{(pid=120863) load_cells /opt/stack/nova/nova/context.py:464}} Aug 27 02:21:40.760483 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_port_service_uri = None {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.760867 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] vmware.task_poll_interval = 0.5 {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.761447 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] vmware.use_linked_clone = True {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.761996 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_keymap = en-us {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.762541 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_port = 5900 {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.763074 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_port_total = 10000 {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.763692 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] vnc.auth_schemes = ['none'] {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.764215 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] vnc.enabled = True {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.764575 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-0a2732d3-b585-470e-97cc-a0a3ddcec5ff None None] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:21:40.765069 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-0a2732d3-b585-470e-97cc-a0a3ddcec5ff None None] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:21:40.765225 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_base_url = http://127.0.0.1:6080/vnc_auto.html {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.767131 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_host = 0.0.0.0 {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.767131 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_port = 6080 {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.767131 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] vnc.server_listen = 127.0.0.1 {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.767609 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-0a2732d3-b585-470e-97cc-a0a3ddcec5ff None None] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:21:40.767753 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] vnc.server_proxyclient_address = 127.0.0.1 {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.767906 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-0a2732d3-b585-470e-97cc-a0a3ddcec5ff None None] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:21:40.768061 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_ca_certs = None {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.768186 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_client_cert = None {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.768479 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-0a2732d3-b585-470e-97cc-a0a3ddcec5ff None None] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:21:40.768592 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_client_key = None {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.768865 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_compute_service_check_for_ffu = False {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.769211 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_fallback_pcpu_query = False {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.769786 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_group_policy_check_upcall = False {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.770231 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_libvirt_livesnapshot = False {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.770231 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_rootwrap = False {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.770877 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.enable_numa_live_migration = False {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.771491 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-0a2732d3-b585-470e-97cc-a0a3ddcec5ff None None] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:21:40.771655 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.enable_qemu_monitor_announce_self = False {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.771655 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.ensure_libvirt_rbd_instance_dir_cleanup = False {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.771655 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.handle_virt_lifecycle_events = True {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.771655 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.libvirt_disable_apic = False {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.772186 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.never_download_image_if_on_rbd = False {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.772678 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO dbcounter [None req-0a2732d3-b585-470e-97cc-a0a3ddcec5ff None None] Registered counter for database nova_cell0 Aug 27 02:21:40.772863 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.reserve_disk_resource_for_image_cache = False {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.773141 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.skip_cpu_compare_on_dest = False {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.773862 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.skip_hypervisor_version_check_on_lm = False {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.774107 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.unified_limits_count_pcpu_as_vcpu = False {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.774599 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.wait_for_vif_plugged_event_during_hard_reboot = [] {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.774813 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.api_paste_config = /etc/nova/api-paste.ini {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.775104 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.client_socket_timeout = 900 {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.775470 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.default_pool_size = 1000 {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.776045 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.keep_alive = True {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.776568 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.max_header_line = 16384 {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.776894 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.secure_proxy_ssl_header = None {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.777300 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_ca_file = None {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.777556 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_cert_file = None {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.777858 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_key_file = None {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.778183 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.tcp_keepidle = 600 {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.778527 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.wsgi_log_format = %(client_ip)s "%(request_line)s" status: %(status_code)s len: %(body_length)s time: %(wall_seconds).7f {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.778842 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] zvm.ca_file = None {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.779188 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] zvm.cloud_connector_url = None {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.779541 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO dbcounter [None req-0a2732d3-b585-470e-97cc-a0a3ddcec5ff None None] Registered counter for database nova_cell1 Aug 27 02:21:40.779739 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] zvm.image_tmp_path = /opt/stack/data/nova/images {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.779890 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] zvm.reachable_timeout = 300 {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.780182 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] oslo_versionedobjects.fatal_exception_format_errors = False {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.780510 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.780827 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.781396 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_process_name = {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.781735 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.782209 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.782373 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.enforce_new_defaults = False {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.782879 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.enforce_scope = False {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.783397 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_default_rule = default {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.783640 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.783972 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_file = policy.yaml {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.784470 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.784942 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.785528 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.786220 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.786678 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.787218 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.auth_section = None {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.787707 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.auth_type = None {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.788026 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.cafile = None {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.788290 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_db.sqlalchemy.engines [None req-0a2732d3-b585-470e-97cc-a0a3ddcec5ff None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=120863) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Aug 27 02:21:40.788514 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.certfile = None {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.788833 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.collect_timing = False {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.789517 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.connect_retries = None {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.789769 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.connect_retry_delay = None {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.790333 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.endpoint_id = None {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.790891 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.endpoint_override = None {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.791469 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.insecure = False {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.791934 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.keyfile = None {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.792408 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.max_version = None {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.793143 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_db.sqlalchemy.engines [None req-0a2732d3-b585-470e-97cc-a0a3ddcec5ff None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=120863) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Aug 27 02:21:40.793405 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.min_version = None {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.793405 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.region_name = None {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.793852 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.service_name = None {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.794092 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.service_type = None {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.794521 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.split_loggers = False {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.794838 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.status_code_retries = None {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.795242 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.status_code_retry_delay = None {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.795559 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.timeout = None {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.795875 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.valid_interfaces = None {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.796467 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.version = None {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.796887 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.797203 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] profiler.connection_string = messaging:// {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.797651 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] profiler.enabled = False {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.797906 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG dbcounter [-] [120863] Writer thread running {{(pid=120863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Aug 27 02:21:40.798145 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_doc_type = notification {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.798525 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_scroll_size = 10000 {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.798848 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_scroll_time = 2m {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.799248 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] profiler.filter_error_trace = False {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.799755 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] profiler.hmac_keys = SECRET_KEY {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.800229 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] profiler.sentinel_service_name = mymaster {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.800811 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] profiler.socket_timeout = 0.1 {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.801423 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] profiler.trace_sqlalchemy = False {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.802049 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.amqp_auto_delete = False {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.802637 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.amqp_durable_queues = False {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.803285 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.conn_pool_min_size = 2 {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.803689 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.conn_pool_ttl = 1200 {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.804276 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.direct_mandatory_flag = True {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.804805 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.enable_cancel_on_failover = False {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.805417 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_in_pthread = False {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.806018 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_rate = 3 {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.807174 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_timeout_threshold = 600 {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.807174 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_compression = None {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.807691 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_failover_strategy = round-robin {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.808694 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_missing_consumer_retry_timeout = 60 {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.808912 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_reconnect_delay = 1.0 {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.809245 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_ha_queues = False {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.809788 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_interval_max = 30 {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.810372 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_login_method = AMQPLAIN {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.810929 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_qos_prefetch_count = 0 {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.811508 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quorum_delivery_limit = 0 {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.812083 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quorum_queue = False {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.812656 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quroum_max_memory_bytes = 0 {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.813445 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quroum_max_memory_length = 0 {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.813845 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_retry_backoff = 2 {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.814426 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_retry_interval = 1 {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.814953 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_transient_queues_ttl = 1800 {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.815433 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rpc_conn_pool_size = 30 {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.815870 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl = False {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.816478 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_ca_file = {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.817057 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_cert_file = {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.817648 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_enforce_fips_mode = False {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.818227 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_key_file = {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.818610 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_version = {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.819145 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.driver = ['messagingv2'] {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.819477 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.retry = -1 {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.819902 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.topics = ['notifications'] {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.820269 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.transport_url = **** {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.822165 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.file_event_handler = None {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.822165 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.file_event_handler_interval = 1 {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.822165 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.log_dir = None {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 27 02:21:40.831595 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi_app [-] ******************************************************************************** {{(pid=120864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Aug 27 02:21:40.948137 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG dbcounter [-] [120863] Writer thread running {{(pid=120863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Aug 27 02:21:40.959693 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO dbcounter [None req-0a2732d3-b585-470e-97cc-a0a3ddcec5ff None None] Registered counter for database nova_cell0 Aug 27 02:21:40.968116 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_db.sqlalchemy.engines [None req-0a2732d3-b585-470e-97cc-a0a3ddcec5ff None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=120863) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Aug 27 02:21:40.973724 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG dbcounter [-] [120863] Writer thread running {{(pid=120863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Aug 27 02:21:41.009605 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO dbcounter [None req-eb6ce849-401e-40cd-aeef-063a5190035c None None] Registered counter for database nova_api Aug 27 02:21:41.030644 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_db.sqlalchemy.engines [None req-eb6ce849-401e-40cd-aeef-063a5190035c None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=120864) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Aug 27 02:21:41.038063 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG dbcounter [-] [120864] Writer thread running {{(pid=120864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Aug 27 02:21:41.098235 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.context [None req-eb6ce849-401e-40cd-aeef-063a5190035c None None] Found 2 cells: 00000000-0000-0000-0000-000000000000(cell0),41dc7c6b-010c-43dc-bd7e-0984fcde0c35(cell1) {{(pid=120864) load_cells /opt/stack/nova/nova/context.py:464}} Aug 27 02:21:41.102174 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-eb6ce849-401e-40cd-aeef-063a5190035c None None] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:21:41.102756 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-eb6ce849-401e-40cd-aeef-063a5190035c None None] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:21:41.104881 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-eb6ce849-401e-40cd-aeef-063a5190035c None None] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:21:41.105658 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-eb6ce849-401e-40cd-aeef-063a5190035c None None] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:21:41.106111 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-eb6ce849-401e-40cd-aeef-063a5190035c None None] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:21:41.108133 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-eb6ce849-401e-40cd-aeef-063a5190035c None None] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:21:41.110193 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO dbcounter [None req-eb6ce849-401e-40cd-aeef-063a5190035c None None] Registered counter for database nova_cell0 Aug 27 02:21:41.114336 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO dbcounter [None req-eb6ce849-401e-40cd-aeef-063a5190035c None None] Registered counter for database nova_cell1 Aug 27 02:21:41.121992 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_db.sqlalchemy.engines [None req-eb6ce849-401e-40cd-aeef-063a5190035c None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=120864) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Aug 27 02:21:41.124186 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_db.sqlalchemy.engines [None req-eb6ce849-401e-40cd-aeef-063a5190035c None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=120864) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Aug 27 02:21:41.128919 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG dbcounter [-] [120864] Writer thread running {{(pid=120864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Aug 27 02:21:41.266627 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG dbcounter [-] [120864] Writer thread running {{(pid=120864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Aug 27 02:21:41.277024 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO dbcounter [None req-eb6ce849-401e-40cd-aeef-063a5190035c None None] Registered counter for database nova_cell0 Aug 27 02:21:41.284810 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_db.sqlalchemy.engines [None req-eb6ce849-401e-40cd-aeef-063a5190035c None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=120864) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Aug 27 02:21:41.291351 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG dbcounter [-] [120864] Writer thread running {{(pid=120864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Aug 27 02:21:41.299530 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: WARNING keystonemiddleware.auth_token [None req-0a2732d3-b585-470e-97cc-a0a3ddcec5ff None None] AuthToken middleware is set with keystone_authtoken.service_token_roles_required set to False. This is backwards compatible but deprecated behaviour. Please set this to True. Aug 27 02:21:41.307287 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: WARNING keystonemiddleware.auth_token [None req-0a2732d3-b585-470e-97cc-a0a3ddcec5ff None None] Configuring www_authenticate_uri to point to the public identity endpoint is required; clients may not be able to authenticate against an admin endpoint Aug 27 02:21:41.625838 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: WARNING keystonemiddleware.auth_token [None req-eb6ce849-401e-40cd-aeef-063a5190035c None None] AuthToken middleware is set with keystone_authtoken.service_token_roles_required set to False. This is backwards compatible but deprecated behaviour. Please set this to True. Aug 27 02:21:41.633459 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: WARNING keystonemiddleware.auth_token [None req-eb6ce849-401e-40cd-aeef-063a5190035c None None] Configuring www_authenticate_uri to point to the public identity endpoint is required; clients may not be able to authenticate against an admin endpoint Aug 27 02:21:41.659912 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: WARNING keystonemiddleware.auth_token [None req-0a2732d3-b585-470e-97cc-a0a3ddcec5ff None None] AuthToken middleware is set with keystone_authtoken.service_token_roles_required set to False. This is backwards compatible but deprecated behaviour. Please set this to True. Aug 27 02:21:41.663831 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: WARNING keystonemiddleware.auth_token [None req-0a2732d3-b585-470e-97cc-a0a3ddcec5ff None None] Configuring www_authenticate_uri to point to the public identity endpoint is required; clients may not be able to authenticate against an admin endpoint Aug 27 02:21:41.670079 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x563e575eb9c0 pid: 120863 (default app) Aug 27 02:21:41.990891 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: WARNING keystonemiddleware.auth_token [None req-eb6ce849-401e-40cd-aeef-063a5190035c None None] AuthToken middleware is set with keystone_authtoken.service_token_roles_required set to False. This is backwards compatible but deprecated behaviour. Please set this to True. Aug 27 02:21:41.998111 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: WARNING keystonemiddleware.auth_token [None req-eb6ce849-401e-40cd-aeef-063a5190035c None None] Configuring www_authenticate_uri to point to the public identity endpoint is required; clients may not be able to authenticate against an admin endpoint Aug 27 02:21:42.004140 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: WSGI app 0 (mountpoint='') ready in 7 seconds on interpreter 0x563e575eb9c0 pid: 120864 (default app) Aug 27 02:22:20.528696 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG dbcounter [-] [120863] Writing DB stats nova_api:SELECT=3 {{(pid=120863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 27 02:22:20.531493 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG dbcounter [-] [120863] Writing DB stats nova_cell0:SELECT=3 {{(pid=120863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 27 02:22:20.532485 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG dbcounter [-] [120863] Writing DB stats nova_cell1:SELECT=3 {{(pid=120863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 27 02:22:20.533475 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG dbcounter [-] [120863] Writing DB stats nova_cell0:SELECT=3 {{(pid=120863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 27 02:22:23.250122 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-8316087c-47ea-4d5d-aa82-730e0d494e86 tempest-VolumesSnapshotsTestJSON-858074346 tempest-VolumesSnapshotsTestJSON-858074346-project] Action: 'create', calling method: .version_select at 0x7f6b1c0cf040>, body: {"volume": {"size": 1, "display_name": "tempest-VolumesSnapshotsTestJSON-volume-949876907"}} {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 27 02:22:26.266861 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-8316087c-47ea-4d5d-aa82-730e0d494e86 tempest-VolumesSnapshotsTestJSON-858074346 tempest-VolumesSnapshotsTestJSON-858074346-project] 10.222.0.52 "POST /compute/v2.1/os-volumes" status: 200 len: 344 microversion: 2.1 time: 5.751781 Aug 27 02:22:26.269202 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 1/1] 10.222.0.52 () {60 vars in 1155 bytes} [Sat Aug 27 02:22:20 2022] POST /compute/v2.1/os-volumes => generated 344 bytes in 5756 msecs (HTTP/1.1 200) 10 headers in 448 bytes (1 switches on core 0) Aug 27 02:22:26.996157 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-c4c987fb-3443-4b31-bb2c-5f3055ef9b69 tempest-VolumesSnapshotsTestJSON-858074346 tempest-VolumesSnapshotsTestJSON-858074346-project] Calling method '.version_select at 0x7f6b1c232040>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:22:27.061646 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG dbcounter [-] [120864] Writing DB stats nova_api:SELECT=3 {{(pid=120864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 27 02:22:27.070537 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG dbcounter [-] [120864] Writing DB stats nova_cell0:SELECT=3 {{(pid=120864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 27 02:22:27.076518 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG dbcounter [-] [120864] Writing DB stats nova_cell1:SELECT=3 {{(pid=120864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 27 02:22:27.077619 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG dbcounter [-] [120864] Writing DB stats nova_cell0:SELECT=3 {{(pid=120864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 27 02:22:28.174719 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-c4c987fb-3443-4b31-bb2c-5f3055ef9b69 tempest-VolumesSnapshotsTestJSON-858074346 tempest-VolumesSnapshotsTestJSON-858074346-project] 10.222.0.52 "GET /compute/v2.1/os-volumes/41f7ec3a-cbc1-4f12-8eb3-b34b1f334cca" status: 200 len: 345 microversion: 2.1 time: 1.893652 Aug 27 02:22:28.183076 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 1/2] 10.222.0.52 () {58 vars in 1245 bytes} [Sat Aug 27 02:22:26 2022] GET /compute/v2.1/os-volumes/41f7ec3a-cbc1-4f12-8eb3-b34b1f334cca => generated 345 bytes in 1896 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 27 02:22:28.190979 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-77b8bb91-d15d-47dc-9efb-e5012da58200 tempest-VolumesSnapshotsTestJSON-858074346 tempest-VolumesSnapshotsTestJSON-858074346-project] Action: 'create', calling method: .version_select at 0x7f6b1c022e50>, body: {"snapshot": {"volume_id": "41f7ec3a-cbc1-4f12-8eb3-b34b1f334cca", "display_name": "tempest-VolumesSnapshotsTestJSON-Snapshot-1371512441"}} {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 27 02:22:28.620893 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-77b8bb91-d15d-47dc-9efb-e5012da58200 tempest-VolumesSnapshotsTestJSON-858074346 tempest-VolumesSnapshotsTestJSON-858074346-project] 10.222.0.52 "POST /compute/v2.1/os-snapshots" status: 200 len: 287 microversion: 2.1 time: 0.434893 Aug 27 02:22:28.624098 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 2/3] 10.222.0.52 () {60 vars in 1162 bytes} [Sat Aug 27 02:22:28 2022] POST /compute/v2.1/os-snapshots => generated 287 bytes in 439 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 27 02:22:28.634825 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-27b2d148-6927-4d25-b75c-8079bd817c55 tempest-VolumesSnapshotsTestJSON-858074346 tempest-VolumesSnapshotsTestJSON-858074346-project] Calling method '.version_select at 0x7f6b1c005e50>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:22:28.703891 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-27b2d148-6927-4d25-b75c-8079bd817c55 tempest-VolumesSnapshotsTestJSON-858074346 tempest-VolumesSnapshotsTestJSON-858074346-project] 10.222.0.52 "GET /compute/v2.1/os-snapshots/32064ba5-791a-467e-a835-6b782c963b35" status: 200 len: 287 microversion: 2.1 time: 0.074493 Aug 27 02:22:28.705254 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 2/4] 10.222.0.52 () {58 vars in 1251 bytes} [Sat Aug 27 02:22:28 2022] GET /compute/v2.1/os-snapshots/32064ba5-791a-467e-a835-6b782c963b35 => generated 287 bytes in 77 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 27 02:22:28.719845 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-6abe67bb-de46-48f8-a3e8-e31c8fc5eef1 tempest-VolumesSnapshotsTestJSON-858074346 tempest-VolumesSnapshotsTestJSON-858074346-project] Calling method '.version_select at 0x7f6b1bfbd040>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:22:28.834692 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-6abe67bb-de46-48f8-a3e8-e31c8fc5eef1 tempest-VolumesSnapshotsTestJSON-858074346 tempest-VolumesSnapshotsTestJSON-858074346-project] 10.222.0.52 "GET /compute/v2.1/os-snapshots" status: 200 len: 290 microversion: 2.1 time: 0.122140 Aug 27 02:22:28.835245 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 3/5] 10.222.0.52 () {58 vars in 1140 bytes} [Sat Aug 27 02:22:28 2022] GET /compute/v2.1/os-snapshots => generated 290 bytes in 124 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 27 02:22:28.848164 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-e0d29365-d10f-4db4-9e99-7fd57b38b3dc tempest-VolumesSnapshotsTestJSON-858074346 tempest-VolumesSnapshotsTestJSON-858074346-project] Calling method '.version_select at 0x7f6b1c005ca0>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:22:28.896048 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-e0d29365-d10f-4db4-9e99-7fd57b38b3dc tempest-VolumesSnapshotsTestJSON-858074346 tempest-VolumesSnapshotsTestJSON-858074346-project] 10.222.0.52 "GET /compute/v2.1/os-snapshots/32064ba5-791a-467e-a835-6b782c963b35" status: 200 len: 287 microversion: 2.1 time: 0.053081 Aug 27 02:22:28.897157 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 3/6] 10.222.0.52 () {58 vars in 1251 bytes} [Sat Aug 27 02:22:28 2022] GET /compute/v2.1/os-snapshots/32064ba5-791a-467e-a835-6b782c963b35 => generated 287 bytes in 55 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Aug 27 02:22:29.909075 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-7c132d6f-efb8-45ef-a8fc-d47afc87ee94 tempest-VolumesSnapshotsTestJSON-858074346 tempest-VolumesSnapshotsTestJSON-858074346-project] Calling method '.version_select at 0x7f6b1c036c10>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:22:29.962362 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-7c132d6f-efb8-45ef-a8fc-d47afc87ee94 tempest-VolumesSnapshotsTestJSON-858074346 tempest-VolumesSnapshotsTestJSON-858074346-project] 10.222.0.52 "GET /compute/v2.1/os-snapshots/32064ba5-791a-467e-a835-6b782c963b35" status: 200 len: 287 microversion: 2.1 time: 0.056803 Aug 27 02:22:29.967096 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 4/7] 10.222.0.52 () {58 vars in 1251 bytes} [Sat Aug 27 02:22:29 2022] GET /compute/v2.1/os-snapshots/32064ba5-791a-467e-a835-6b782c963b35 => generated 287 bytes in 58 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 27 02:22:30.976795 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-c9506f6a-7cae-4977-abfe-11f471c52edc tempest-VolumesSnapshotsTestJSON-858074346 tempest-VolumesSnapshotsTestJSON-858074346-project] Calling method '.version_select at 0x7f6b1bfd1670>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:22:31.034137 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-c9506f6a-7cae-4977-abfe-11f471c52edc tempest-VolumesSnapshotsTestJSON-858074346 tempest-VolumesSnapshotsTestJSON-858074346-project] 10.222.0.52 "GET /compute/v2.1/os-snapshots/32064ba5-791a-467e-a835-6b782c963b35" status: 200 len: 287 microversion: 2.1 time: 0.062787 Aug 27 02:22:31.035605 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 4/8] 10.222.0.52 () {58 vars in 1251 bytes} [Sat Aug 27 02:22:30 2022] GET /compute/v2.1/os-snapshots/32064ba5-791a-467e-a835-6b782c963b35 => generated 287 bytes in 65 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Aug 27 02:22:31.329987 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-893e6954-cd56-4f1e-a7df-bff8bc63eb67 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-ServerRescueNegativeTestJSON-server-1074384467", "imageRef": "9fc30b36-b827-46f9-892d-30e808bc2b8a", "flavorRef": "452", "adminPass": "***", "networks": [{"uuid": "}]}} {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 27 02:22:31.477004 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.network.neutron [None req-893e6954-cd56-4f1e-a7df-bff8bc63eb67 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] validate_networks() for [('8ba10e7b-62f2-400d-a225-3160ea7a308c', None, None, None, None, None)] {{(pid=120863) validate_networks /opt/stack/nova/nova/network/neutron.py:2626}} Aug 27 02:22:32.055109 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-39541e63-d16c-4a37-98ad-76660a60dc17 tempest-VolumesSnapshotsTestJSON-858074346 tempest-VolumesSnapshotsTestJSON-858074346-project] Calling method '.version_select at 0x7f6b1bfd1d30>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:22:32.155682 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-39541e63-d16c-4a37-98ad-76660a60dc17 tempest-VolumesSnapshotsTestJSON-858074346 tempest-VolumesSnapshotsTestJSON-858074346-project] 10.222.0.52 "GET /compute/v2.1/os-snapshots/32064ba5-791a-467e-a835-6b782c963b35" status: 200 len: 287 microversion: 2.1 time: 0.106822 Aug 27 02:22:32.163592 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 5/9] 10.222.0.52 () {58 vars in 1251 bytes} [Sat Aug 27 02:22:32 2022] GET /compute/v2.1/os-snapshots/32064ba5-791a-467e-a835-6b782c963b35 => generated 287 bytes in 120 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 27 02:22:33.169098 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-b3f839b0-b360-45ca-932a-0991b1dcc039 tempest-VolumesSnapshotsTestJSON-858074346 tempest-VolumesSnapshotsTestJSON-858074346-project] Calling method '.version_select at 0x7f6b1bfd1c10>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:22:33.269302 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-b3f839b0-b360-45ca-932a-0991b1dcc039 tempest-VolumesSnapshotsTestJSON-858074346 tempest-VolumesSnapshotsTestJSON-858074346-project] 10.222.0.52 "GET /compute/v2.1/os-snapshots/32064ba5-791a-467e-a835-6b782c963b35" status: 200 len: 287 microversion: 2.1 time: 0.105043 Aug 27 02:22:33.275175 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 6/10] 10.222.0.52 () {58 vars in 1251 bytes} [Sat Aug 27 02:22:33 2022] GET /compute/v2.1/os-snapshots/32064ba5-791a-467e-a835-6b782c963b35 => generated 287 bytes in 111 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 27 02:22:33.453631 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.virt.hardware [None req-893e6954-cd56-4f1e-a7df-bff8bc63eb67 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Flavor limits 0:0:0 {{(pid=120863) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Aug 27 02:22:33.454123 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.virt.hardware [None req-893e6954-cd56-4f1e-a7df-bff8bc63eb67 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Image limits 0:0:0 {{(pid=120863) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Aug 27 02:22:33.454513 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.virt.hardware [None req-893e6954-cd56-4f1e-a7df-bff8bc63eb67 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Flavor pref 0:0:0 {{(pid=120863) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Aug 27 02:22:33.454916 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.virt.hardware [None req-893e6954-cd56-4f1e-a7df-bff8bc63eb67 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Image pref 0:0:0 {{(pid=120863) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Aug 27 02:22:33.455328 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.virt.hardware [None req-893e6954-cd56-4f1e-a7df-bff8bc63eb67 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=120863) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Aug 27 02:22:33.494782 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-893e6954-cd56-4f1e-a7df-bff8bc63eb67 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:22:33.500747 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-893e6954-cd56-4f1e-a7df-bff8bc63eb67 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.006s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:22:33.501228 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-893e6954-cd56-4f1e-a7df-bff8bc63eb67 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:22:33.502020 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-893e6954-cd56-4f1e-a7df-bff8bc63eb67 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:22:33.506667 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-893e6954-cd56-4f1e-a7df-bff8bc63eb67 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:22:33.507138 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-893e6954-cd56-4f1e-a7df-bff8bc63eb67 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:22:33.559794 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.quota [None req-893e6954-cd56-4f1e-a7df-bff8bc63eb67 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Getting quotas for project 880ac4003248441c809a76cfd1b221eb. Resources: {'cores', 'ram', 'instances'} {{(pid=120863) _get_quotas /opt/stack/nova/nova/quota.py:393}} Aug 27 02:22:33.587609 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.quota [None req-893e6954-cd56-4f1e-a7df-bff8bc63eb67 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Getting quotas for user d945d512ae034ec08838d0a3afde2429 and project 880ac4003248441c809a76cfd1b221eb. Resources: {'cores', 'ram', 'instances'} {{(pid=120863) _get_quotas /opt/stack/nova/nova/quota.py:383}} Aug 27 02:22:33.624981 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-893e6954-cd56-4f1e-a7df-bff8bc63eb67 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Going to run 1 instances... {{(pid=120863) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Aug 27 02:22:33.628062 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-893e6954-cd56-4f1e-a7df-bff8bc63eb67 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] [instance: e2bf7024-0779-4e08-844a-43b78187e770] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='9fc30b36-b827-46f9-892d-30e808bc2b8a',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=120863) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Aug 27 02:22:33.777107 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-e76ae622-5005-46f2-8efe-20f7a87b54cf tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Action: 'create', calling method: .version_select at 0x7f6b1bfd15e0>, body: {"keypair": {"name": "tempest-keypair-1257325292", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBJkQ/eXOK/D/Zxis9RBWsxrXVZupUPsHClU4VkGY/SRl28YnQQFIpA35Bul36BWN6SQy4xhYccPCfz7oDFZ7UOI+62ioNjwR6Q2AdBaIuCSKfjBBEDKHMtjSiONyYQLEgg=="}} {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 27 02:22:33.809329 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-893e6954-cd56-4f1e-a7df-bff8bc63eb67 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] 10.222.0.52 "POST /compute/v2.1/servers" status: 202 len: 383 microversion: 2.1 time: 2.741550 Aug 27 02:22:33.810256 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 5/11] 10.222.0.52 () {60 vars in 1147 bytes} [Sat Aug 27 02:22:31 2022] POST /compute/v2.1/servers => generated 383 bytes in 2744 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Aug 27 02:22:33.824630 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-32649e69-0db1-416a-a464-b02a60ef69b2 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:22:33.825660 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-32649e69-0db1-416a-a464-b02a60ef69b2 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] [instance: e2bf7024-0779-4e08-844a-43b78187e770] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:22:33.880614 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.quota [None req-e76ae622-5005-46f2-8efe-20f7a87b54cf tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Getting quotas for project 5d5ec8450c0941d18d63558baab54737. Resources: {'key_pairs'} {{(pid=120864) _get_quotas /opt/stack/nova/nova/quota.py:393}} Aug 27 02:22:33.907713 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.quota [None req-e76ae622-5005-46f2-8efe-20f7a87b54cf tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Getting quotas for user 26a86351583d49d090b3db09f127d49f and project 5d5ec8450c0941d18d63558baab54737. Resources: {'key_pairs'} {{(pid=120864) _get_quotas /opt/stack/nova/nova/quota.py:383}} Aug 27 02:22:33.976487 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-e76ae622-5005-46f2-8efe-20f7a87b54cf tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] 10.222.0.52 "POST /compute/v2.1/os-keypairs" status: 200 len: 386 microversion: 2.1 time: 0.464674 Aug 27 02:22:33.977793 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 7/12] 10.222.0.52 () {60 vars in 1159 bytes} [Sat Aug 27 02:22:33 2022] POST /compute/v2.1/os-keypairs => generated 386 bytes in 467 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Aug 27 02:22:34.039252 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-32649e69-0db1-416a-a464-b02a60ef69b2 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd945d512ae034ec08838d0a3afde2429', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '880ac4003248441c809a76cfd1b221eb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:22:34.069982 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-32649e69-0db1-416a-a464-b02a60ef69b2 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] 10.222.0.52 "GET /compute/v2.1/servers/e2bf7024-0779-4e08-844a-43b78187e770" status: 200 len: 1222 microversion: 2.1 time: 0.248760 Aug 27 02:22:34.074770 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 6/13] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:22:33 2022] GET /compute/v2.1/servers/e2bf7024-0779-4e08-844a-43b78187e770 => generated 1222 bytes in 256 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 27 02:22:34.284426 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-70a5df4b-c3c6-4b75-86d8-c183c3cba4f8 tempest-VolumesSnapshotsTestJSON-858074346 tempest-VolumesSnapshotsTestJSON-858074346-project] Calling method '.version_select at 0x7f6b1bf78160>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:22:34.329029 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-70a5df4b-c3c6-4b75-86d8-c183c3cba4f8 tempest-VolumesSnapshotsTestJSON-858074346 tempest-VolumesSnapshotsTestJSON-858074346-project] 10.222.0.52 "GET /compute/v2.1/os-snapshots/32064ba5-791a-467e-a835-6b782c963b35" status: 200 len: 287 microversion: 2.1 time: 0.049700 Aug 27 02:22:34.329849 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 8/14] 10.222.0.52 () {58 vars in 1251 bytes} [Sat Aug 27 02:22:34 2022] GET /compute/v2.1/os-snapshots/32064ba5-791a-467e-a835-6b782c963b35 => generated 287 bytes in 51 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 27 02:22:34.474331 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-e3b3efc5-1688-487e-b162-0a1aa50d2dd1 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-ServerRescueNegativeTestJSON-server-1333478733", "imageRef": "9fc30b36-b827-46f9-892d-30e808bc2b8a", "flavorRef": "452", "adminPass": "***", "networks": [{"uuid": "}]}} {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 27 02:22:34.587612 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.network.neutron [None req-e3b3efc5-1688-487e-b162-0a1aa50d2dd1 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] validate_networks() for [('8ba10e7b-62f2-400d-a225-3160ea7a308c', None, None, None, None, None)] {{(pid=120863) validate_networks /opt/stack/nova/nova/network/neutron.py:2626}} Aug 27 02:22:35.643811 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.virt.hardware [None req-e3b3efc5-1688-487e-b162-0a1aa50d2dd1 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Flavor limits 0:0:0 {{(pid=120863) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Aug 27 02:22:35.645079 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.virt.hardware [None req-e3b3efc5-1688-487e-b162-0a1aa50d2dd1 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Image limits 0:0:0 {{(pid=120863) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Aug 27 02:22:35.646958 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.virt.hardware [None req-e3b3efc5-1688-487e-b162-0a1aa50d2dd1 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Flavor pref 0:0:0 {{(pid=120863) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Aug 27 02:22:35.647986 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.virt.hardware [None req-e3b3efc5-1688-487e-b162-0a1aa50d2dd1 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Image pref 0:0:0 {{(pid=120863) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Aug 27 02:22:35.648612 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.virt.hardware [None req-e3b3efc5-1688-487e-b162-0a1aa50d2dd1 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=120863) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Aug 27 02:22:35.673157 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-e3b3efc5-1688-487e-b162-0a1aa50d2dd1 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:22:35.674800 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-e3b3efc5-1688-487e-b162-0a1aa50d2dd1 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.006s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:22:35.677156 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-d4aeaae3-5de9-4bde-b2f3-099f08c060c2 tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-DeleteServersTestJSON-server-296953754", "imageRef": "9fc30b36-b827-46f9-892d-30e808bc2b8a", "flavorRef": "452", "networks": [{"uuid": "1d7da9b1-bc52-4897-b0d0-f4970cc43ec9"}]}} {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 27 02:22:35.678393 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-e3b3efc5-1688-487e-b162-0a1aa50d2dd1 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:22:35.681578 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-e3b3efc5-1688-487e-b162-0a1aa50d2dd1 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:22:35.684454 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-e3b3efc5-1688-487e-b162-0a1aa50d2dd1 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:22:35.685154 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-e3b3efc5-1688-487e-b162-0a1aa50d2dd1 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:22:35.733683 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.quota [None req-e3b3efc5-1688-487e-b162-0a1aa50d2dd1 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Getting quotas for project 880ac4003248441c809a76cfd1b221eb. Resources: {'cores', 'ram', 'instances'} {{(pid=120863) _get_quotas /opt/stack/nova/nova/quota.py:393}} Aug 27 02:22:35.773105 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.quota [None req-e3b3efc5-1688-487e-b162-0a1aa50d2dd1 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Getting quotas for user d945d512ae034ec08838d0a3afde2429 and project 880ac4003248441c809a76cfd1b221eb. Resources: {'cores', 'ram', 'instances'} {{(pid=120863) _get_quotas /opt/stack/nova/nova/quota.py:383}} Aug 27 02:22:35.812633 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-e3b3efc5-1688-487e-b162-0a1aa50d2dd1 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Going to run 1 instances... {{(pid=120863) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Aug 27 02:22:35.814875 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-e3b3efc5-1688-487e-b162-0a1aa50d2dd1 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] [instance: 05bed8e9-f530-418a-9819-4f65dabdcda0] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='9fc30b36-b827-46f9-892d-30e808bc2b8a',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=120863) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Aug 27 02:22:35.898376 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.network.neutron [None req-d4aeaae3-5de9-4bde-b2f3-099f08c060c2 tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] validate_networks() for [('1d7da9b1-bc52-4897-b0d0-f4970cc43ec9', None, None, None, None, None)] {{(pid=120864) validate_networks /opt/stack/nova/nova/network/neutron.py:2626}} Aug 27 02:22:35.986958 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-e3b3efc5-1688-487e-b162-0a1aa50d2dd1 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] 10.222.0.52 "POST /compute/v2.1/servers" status: 202 len: 383 microversion: 2.1 time: 1.519651 Aug 27 02:22:35.993977 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 7/15] 10.222.0.52 () {60 vars in 1147 bytes} [Sat Aug 27 02:22:34 2022] POST /compute/v2.1/servers => generated 383 bytes in 1523 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Aug 27 02:22:36.005625 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-813bd331-370c-4ee0-b7a4-98ac5bcfc82f tempest-VolumesSnapshotsTestJSON-858074346 tempest-VolumesSnapshotsTestJSON-858074346-project] Calling method '.version_select at 0x7f6b1bfbdee0>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:22:36.076716 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-813bd331-370c-4ee0-b7a4-98ac5bcfc82f tempest-VolumesSnapshotsTestJSON-858074346 tempest-VolumesSnapshotsTestJSON-858074346-project] 10.222.0.52 "GET /compute/v2.1/os-snapshots/32064ba5-791a-467e-a835-6b782c963b35" status: 200 len: 287 microversion: 2.1 time: 0.080447 Aug 27 02:22:36.077664 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 8/16] 10.222.0.52 () {58 vars in 1251 bytes} [Sat Aug 27 02:22:35 2022] GET /compute/v2.1/os-snapshots/32064ba5-791a-467e-a835-6b782c963b35 => generated 287 bytes in 82 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 27 02:22:36.085724 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-ef0bbf1a-a540-4e74-b51f-83a2fcc4e866 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:22:36.086571 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-ef0bbf1a-a540-4e74-b51f-83a2fcc4e866 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] [instance: 05bed8e9-f530-418a-9819-4f65dabdcda0] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:22:36.557068 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-ef0bbf1a-a540-4e74-b51f-83a2fcc4e866 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd945d512ae034ec08838d0a3afde2429', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '880ac4003248441c809a76cfd1b221eb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:22:36.570741 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-ef0bbf1a-a540-4e74-b51f-83a2fcc4e866 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] 10.222.0.52 "GET /compute/v2.1/servers/05bed8e9-f530-418a-9819-4f65dabdcda0" status: 200 len: 1222 microversion: 2.1 time: 0.490079 Aug 27 02:22:36.575800 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 9/17] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:22:36 2022] GET /compute/v2.1/servers/05bed8e9-f530-418a-9819-4f65dabdcda0 => generated 1222 bytes in 497 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:22:37.090307 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-91af1df2-8128-4710-939b-267bca98d277 tempest-VolumesSnapshotsTestJSON-858074346 tempest-VolumesSnapshotsTestJSON-858074346-project] Calling method '.version_select at 0x7f6b1b67c550>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:22:37.181233 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-91af1df2-8128-4710-939b-267bca98d277 tempest-VolumesSnapshotsTestJSON-858074346 tempest-VolumesSnapshotsTestJSON-858074346-project] 10.222.0.52 "GET /compute/v2.1/os-snapshots/32064ba5-791a-467e-a835-6b782c963b35" status: 200 len: 287 microversion: 2.1 time: 0.095148 Aug 27 02:22:37.182471 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 10/18] 10.222.0.52 () {58 vars in 1251 bytes} [Sat Aug 27 02:22:37 2022] GET /compute/v2.1/os-snapshots/32064ba5-791a-467e-a835-6b782c963b35 => generated 287 bytes in 97 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 27 02:22:37.599052 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-1fe98412-6ed6-4288-a010-5a3858c95571 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:22:37.599720 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-1fe98412-6ed6-4288-a010-5a3858c95571 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] [instance: 05bed8e9-f530-418a-9819-4f65dabdcda0] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:22:37.968647 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-1fe98412-6ed6-4288-a010-5a3858c95571 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd945d512ae034ec08838d0a3afde2429', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '880ac4003248441c809a76cfd1b221eb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:22:37.981305 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-1fe98412-6ed6-4288-a010-5a3858c95571 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] 10.222.0.52 "GET /compute/v2.1/servers/05bed8e9-f530-418a-9819-4f65dabdcda0" status: 200 len: 1222 microversion: 2.1 time: 0.385952 Aug 27 02:22:37.982429 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 11/19] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:22:37 2022] GET /compute/v2.1/servers/05bed8e9-f530-418a-9819-4f65dabdcda0 => generated 1222 bytes in 388 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 27 02:22:38.196736 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-adeea835-cd82-4278-ac83-284b27a2802d tempest-VolumesSnapshotsTestJSON-858074346 tempest-VolumesSnapshotsTestJSON-858074346-project] Calling method '.version_select at 0x7f6b1b67c9d0>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:22:38.235233 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.virt.hardware [None req-d4aeaae3-5de9-4bde-b2f3-099f08c060c2 tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Flavor limits 0:0:0 {{(pid=120864) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Aug 27 02:22:38.235766 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.virt.hardware [None req-d4aeaae3-5de9-4bde-b2f3-099f08c060c2 tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Image limits 0:0:0 {{(pid=120864) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Aug 27 02:22:38.236125 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.virt.hardware [None req-d4aeaae3-5de9-4bde-b2f3-099f08c060c2 tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Flavor pref 0:0:0 {{(pid=120864) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Aug 27 02:22:38.236535 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.virt.hardware [None req-d4aeaae3-5de9-4bde-b2f3-099f08c060c2 tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Image pref 0:0:0 {{(pid=120864) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Aug 27 02:22:38.236948 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.virt.hardware [None req-d4aeaae3-5de9-4bde-b2f3-099f08c060c2 tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=120864) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Aug 27 02:22:38.254785 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-adeea835-cd82-4278-ac83-284b27a2802d tempest-VolumesSnapshotsTestJSON-858074346 tempest-VolumesSnapshotsTestJSON-858074346-project] 10.222.0.52 "GET /compute/v2.1/os-snapshots/32064ba5-791a-467e-a835-6b782c963b35" status: 200 len: 287 microversion: 2.1 time: 0.063716 Aug 27 02:22:38.255486 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 12/20] 10.222.0.52 () {58 vars in 1251 bytes} [Sat Aug 27 02:22:38 2022] GET /compute/v2.1/os-snapshots/32064ba5-791a-467e-a835-6b782c963b35 => generated 287 bytes in 65 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 27 02:22:38.260443 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-d4aeaae3-5de9-4bde-b2f3-099f08c060c2 tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:22:38.261166 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-d4aeaae3-5de9-4bde-b2f3-099f08c060c2 tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:22:38.261767 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-d4aeaae3-5de9-4bde-b2f3-099f08c060c2 tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:22:38.262654 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-d4aeaae3-5de9-4bde-b2f3-099f08c060c2 tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:22:38.263027 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-d4aeaae3-5de9-4bde-b2f3-099f08c060c2 tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:22:38.263436 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-d4aeaae3-5de9-4bde-b2f3-099f08c060c2 tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:22:38.310791 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.quota [None req-d4aeaae3-5de9-4bde-b2f3-099f08c060c2 tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Getting quotas for project f7ef4b44a3f94d599eacce269b06594c. Resources: {'cores', 'ram', 'instances'} {{(pid=120864) _get_quotas /opt/stack/nova/nova/quota.py:393}} Aug 27 02:22:38.315860 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.quota [None req-d4aeaae3-5de9-4bde-b2f3-099f08c060c2 tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Getting quotas for user 55640f750d1b49be9c3f79501a63c1ab and project f7ef4b44a3f94d599eacce269b06594c. Resources: {'cores', 'ram', 'instances'} {{(pid=120864) _get_quotas /opt/stack/nova/nova/quota.py:383}} Aug 27 02:22:38.331311 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-d4aeaae3-5de9-4bde-b2f3-099f08c060c2 tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Going to run 1 instances... {{(pid=120864) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Aug 27 02:22:38.337398 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-d4aeaae3-5de9-4bde-b2f3-099f08c060c2 tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] [instance: fbef28bb-3cdf-4a27-8493-add7806af828] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='9fc30b36-b827-46f9-892d-30e808bc2b8a',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=120864) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Aug 27 02:22:38.355319 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-5716eb54-110f-4768-ade6-5b667820f5e4 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-AttachVolumeNegativeTest-server-360204839", "imageRef": "9fc30b36-b827-46f9-892d-30e808bc2b8a", "flavorRef": "452", "adminPass": "***", "networks": [{"uuid": "134184f4-d632-4173-b77c-1566bf4eb0fb"}], "security_groups": [{"name": "tempest-securitygroup--726596947"}], "key_name": "tempest-keypair-1257325292", "user_data": "}} {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 27 02:22:38.485469 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-d4aeaae3-5de9-4bde-b2f3-099f08c060c2 tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] 10.222.0.52 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 3.323905 Aug 27 02:22:38.499330 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 9/21] 10.222.0.52 () {60 vars in 1147 bytes} [Sat Aug 27 02:22:35 2022] POST /compute/v2.1/servers => generated 380 bytes in 3339 msecs (HTTP/1.1 202) 10 headers in 451 bytes (2 switches on core 0) Aug 27 02:22:38.505928 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-83988278-faed-4edd-8e22-a9855a49dcc6 tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Calling method '>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:22:38.507172 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-83988278-faed-4edd-8e22-a9855a49dcc6 tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] [instance: fbef28bb-3cdf-4a27-8493-add7806af828] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:22:38.611655 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.network.neutron [None req-5716eb54-110f-4768-ade6-5b667820f5e4 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] validate_networks() for [('134184f4-d632-4173-b77c-1566bf4eb0fb', None, None, None, None, None)] {{(pid=120863) validate_networks /opt/stack/nova/nova/network/neutron.py:2626}} Aug 27 02:22:38.705689 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-83988278-faed-4edd-8e22-a9855a49dcc6 tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '55640f750d1b49be9c3f79501a63c1ab', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7ef4b44a3f94d599eacce269b06594c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:22:38.733493 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-83988278-faed-4edd-8e22-a9855a49dcc6 tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] 10.222.0.52 "GET /compute/v2.1/servers/fbef28bb-3cdf-4a27-8493-add7806af828" status: 200 len: 1214 microversion: 2.1 time: 0.228190 Aug 27 02:22:38.748390 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 10/22] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:22:38 2022] GET /compute/v2.1/servers/fbef28bb-3cdf-4a27-8493-add7806af828 => generated 1214 bytes in 248 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:22:39.003335 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-c5faff73-7be1-4c1d-881f-20c5d19589d9 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Calling method '>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:22:39.004212 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-c5faff73-7be1-4c1d-881f-20c5d19589d9 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] [instance: 05bed8e9-f530-418a-9819-4f65dabdcda0] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:22:39.032370 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-c5faff73-7be1-4c1d-881f-20c5d19589d9 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:22:39.034762 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-c5faff73-7be1-4c1d-881f-20c5d19589d9 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:22:39.035422 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-c5faff73-7be1-4c1d-881f-20c5d19589d9 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:22:39.036325 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-c5faff73-7be1-4c1d-881f-20c5d19589d9 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:22:39.038498 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-c5faff73-7be1-4c1d-881f-20c5d19589d9 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:22:39.039109 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-c5faff73-7be1-4c1d-881f-20c5d19589d9 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:22:39.255137 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.virt.hardware [None req-5716eb54-110f-4768-ade6-5b667820f5e4 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Flavor limits 0:0:0 {{(pid=120863) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Aug 27 02:22:39.255137 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.virt.hardware [None req-5716eb54-110f-4768-ade6-5b667820f5e4 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Image limits 0:0:0 {{(pid=120863) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Aug 27 02:22:39.255137 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.virt.hardware [None req-5716eb54-110f-4768-ade6-5b667820f5e4 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Flavor pref 0:0:0 {{(pid=120863) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Aug 27 02:22:39.255137 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.virt.hardware [None req-5716eb54-110f-4768-ade6-5b667820f5e4 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Image pref 0:0:0 {{(pid=120863) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Aug 27 02:22:39.255137 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.virt.hardware [None req-5716eb54-110f-4768-ade6-5b667820f5e4 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=120863) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Aug 27 02:22:39.264774 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-5716eb54-110f-4768-ade6-5b667820f5e4 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:22:39.266046 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-5716eb54-110f-4768-ade6-5b667820f5e4 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:22:39.266873 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-5716eb54-110f-4768-ade6-5b667820f5e4 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:22:39.267708 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-5716eb54-110f-4768-ade6-5b667820f5e4 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:22:39.269253 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-5716eb54-110f-4768-ade6-5b667820f5e4 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:22:39.269808 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-5716eb54-110f-4768-ade6-5b667820f5e4 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:22:39.292926 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.quota [None req-5716eb54-110f-4768-ade6-5b667820f5e4 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Getting quotas for project 5d5ec8450c0941d18d63558baab54737. Resources: {'cores', 'ram', 'instances'} {{(pid=120863) _get_quotas /opt/stack/nova/nova/quota.py:393}} Aug 27 02:22:39.297998 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.quota [None req-5716eb54-110f-4768-ade6-5b667820f5e4 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Getting quotas for user 26a86351583d49d090b3db09f127d49f and project 5d5ec8450c0941d18d63558baab54737. Resources: {'cores', 'ram', 'instances'} {{(pid=120863) _get_quotas /opt/stack/nova/nova/quota.py:383}} Aug 27 02:22:39.306648 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-c5faff73-7be1-4c1d-881f-20c5d19589d9 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd945d512ae034ec08838d0a3afde2429', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '880ac4003248441c809a76cfd1b221eb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:22:39.312677 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-5716eb54-110f-4768-ade6-5b667820f5e4 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Going to run 1 instances... {{(pid=120863) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Aug 27 02:22:39.315833 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-5716eb54-110f-4768-ade6-5b667820f5e4 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] [instance: 2ad09ab6-ef1f-47a2-bf44-6ae046cc8645] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='9fc30b36-b827-46f9-892d-30e808bc2b8a',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=120863) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Aug 27 02:22:39.325407 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-c5faff73-7be1-4c1d-881f-20c5d19589d9 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] 10.222.0.52 "GET /compute/v2.1/servers/05bed8e9-f530-418a-9819-4f65dabdcda0" status: 200 len: 1226 microversion: 2.1 time: 0.325902 Aug 27 02:22:39.327028 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 11/23] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:22:38 2022] GET /compute/v2.1/servers/05bed8e9-f530-418a-9819-4f65dabdcda0 => generated 1226 bytes in 329 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 27 02:22:39.332628 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-b61a2143-c17c-4d86-9d2c-d9998deac8f0 tempest-VolumesSnapshotsTestJSON-858074346 tempest-VolumesSnapshotsTestJSON-858074346-project] Calling method '.version_select at 0x7f6b1b635790>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:22:39.381573 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-b61a2143-c17c-4d86-9d2c-d9998deac8f0 tempest-VolumesSnapshotsTestJSON-858074346 tempest-VolumesSnapshotsTestJSON-858074346-project] 10.222.0.52 "GET /compute/v2.1/os-snapshots/32064ba5-791a-467e-a835-6b782c963b35" status: 200 len: 287 microversion: 2.1 time: 0.052857 Aug 27 02:22:39.382408 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 12/24] 10.222.0.52 () {58 vars in 1251 bytes} [Sat Aug 27 02:22:39 2022] GET /compute/v2.1/os-snapshots/32064ba5-791a-467e-a835-6b782c963b35 => generated 287 bytes in 55 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 27 02:22:39.415357 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-5716eb54-110f-4768-ade6-5b667820f5e4 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] 10.222.0.52 "POST /compute/v2.1/servers" status: 202 len: 401 microversion: 2.1 time: 1.066539 Aug 27 02:22:39.425771 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 13/25] 10.222.0.52 () {60 vars in 1147 bytes} [Sat Aug 27 02:22:38 2022] POST /compute/v2.1/servers => generated 401 bytes in 1078 msecs (HTTP/1.1 202) 10 headers in 451 bytes (2 switches on core 0) Aug 27 02:22:39.426872 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-9ee18aed-01c7-4d6d-8f5a-cbb0721331d0 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Calling method '>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:22:39.427498 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-9ee18aed-01c7-4d6d-8f5a-cbb0721331d0 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] [instance: 2ad09ab6-ef1f-47a2-bf44-6ae046cc8645] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:22:39.555992 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-9ee18aed-01c7-4d6d-8f5a-cbb0721331d0 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '26a86351583d49d090b3db09f127d49f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5d5ec8450c0941d18d63558baab54737', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:22:39.615044 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-9ee18aed-01c7-4d6d-8f5a-cbb0721331d0 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] 10.222.0.52 "GET /compute/v2.1/servers/2ad09ab6-ef1f-47a2-bf44-6ae046cc8645" status: 200 len: 1241 microversion: 2.1 time: 0.186502 Aug 27 02:22:39.617588 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 13/26] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:22:39 2022] GET /compute/v2.1/servers/2ad09ab6-ef1f-47a2-bf44-6ae046cc8645 => generated 1241 bytes in 195 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:22:39.759343 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-827d78c0-1509-4e88-acba-fe3f5df3ce22 tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:22:39.762897 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-827d78c0-1509-4e88-acba-fe3f5df3ce22 tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] [instance: fbef28bb-3cdf-4a27-8493-add7806af828] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:22:39.795081 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-827d78c0-1509-4e88-acba-fe3f5df3ce22 tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:22:39.798616 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-827d78c0-1509-4e88-acba-fe3f5df3ce22 tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:22:39.799999 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-827d78c0-1509-4e88-acba-fe3f5df3ce22 tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:22:39.805613 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-827d78c0-1509-4e88-acba-fe3f5df3ce22 tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:22:39.806105 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-827d78c0-1509-4e88-acba-fe3f5df3ce22 tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:22:39.808309 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-827d78c0-1509-4e88-acba-fe3f5df3ce22 tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:22:40.165861 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-827d78c0-1509-4e88-acba-fe3f5df3ce22 tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '55640f750d1b49be9c3f79501a63c1ab', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7ef4b44a3f94d599eacce269b06594c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:22:40.216983 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-827d78c0-1509-4e88-acba-fe3f5df3ce22 tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] 10.222.0.52 "GET /compute/v2.1/servers/fbef28bb-3cdf-4a27-8493-add7806af828" status: 200 len: 1210 microversion: 2.1 time: 0.465076 Aug 27 02:22:40.218969 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 14/27] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:22:39 2022] GET /compute/v2.1/servers/fbef28bb-3cdf-4a27-8493-add7806af828 => generated 1210 bytes in 468 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:22:40.350535 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-a3ca8352-71a6-460f-a464-38a1af891bfe tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Calling method '>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:22:40.351513 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-a3ca8352-71a6-460f-a464-38a1af891bfe tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] [instance: 05bed8e9-f530-418a-9819-4f65dabdcda0] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:22:40.360691 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-a3ca8352-71a6-460f-a464-38a1af891bfe tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:22:40.361309 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-a3ca8352-71a6-460f-a464-38a1af891bfe tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:22:40.361643 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-a3ca8352-71a6-460f-a464-38a1af891bfe tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:22:40.375877 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-a3ca8352-71a6-460f-a464-38a1af891bfe tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:22:40.378602 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-a3ca8352-71a6-460f-a464-38a1af891bfe tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.015s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:22:40.379495 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-a3ca8352-71a6-460f-a464-38a1af891bfe tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:22:40.394413 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-bdbb5b90-6c39-45a5-95ff-06ed471334fd tempest-VolumesSnapshotsTestJSON-858074346 tempest-VolumesSnapshotsTestJSON-858074346-project] Calling method '.version_select at 0x7f6b1b602940>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:22:40.573200 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-a3ca8352-71a6-460f-a464-38a1af891bfe tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd945d512ae034ec08838d0a3afde2429', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '880ac4003248441c809a76cfd1b221eb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:22:40.590810 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-a3ca8352-71a6-460f-a464-38a1af891bfe tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] 10.222.0.52 "GET /compute/v2.1/servers/05bed8e9-f530-418a-9819-4f65dabdcda0" status: 200 len: 1274 microversion: 2.1 time: 0.245521 Aug 27 02:22:40.593483 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 14/28] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:22:40 2022] GET /compute/v2.1/servers/05bed8e9-f530-418a-9819-4f65dabdcda0 => generated 1274 bytes in 250 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:22:40.637875 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-b490cecb-9b99-43b3-8796-87a9a2fd1c97 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Calling method '>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:22:40.638805 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-b490cecb-9b99-43b3-8796-87a9a2fd1c97 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] [instance: 2ad09ab6-ef1f-47a2-bf44-6ae046cc8645] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:22:40.650833 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-b490cecb-9b99-43b3-8796-87a9a2fd1c97 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:22:40.651543 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-b490cecb-9b99-43b3-8796-87a9a2fd1c97 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:22:40.652165 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-b490cecb-9b99-43b3-8796-87a9a2fd1c97 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:22:40.653097 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-b490cecb-9b99-43b3-8796-87a9a2fd1c97 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:22:40.653857 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-b490cecb-9b99-43b3-8796-87a9a2fd1c97 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:22:40.654659 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-b490cecb-9b99-43b3-8796-87a9a2fd1c97 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:22:40.669255 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-bdbb5b90-6c39-45a5-95ff-06ed471334fd tempest-VolumesSnapshotsTestJSON-858074346 tempest-VolumesSnapshotsTestJSON-858074346-project] 10.222.0.52 "GET /compute/v2.1/os-snapshots/32064ba5-791a-467e-a835-6b782c963b35" status: 200 len: 288 microversion: 2.1 time: 0.278723 Aug 27 02:22:40.669981 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 15/29] 10.222.0.52 () {58 vars in 1251 bytes} [Sat Aug 27 02:22:40 2022] GET /compute/v2.1/os-snapshots/32064ba5-791a-467e-a835-6b782c963b35 => generated 288 bytes in 280 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 27 02:22:40.682495 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-6e280401-96f6-492e-8c44-9f4d8c941710 tempest-VolumesSnapshotsTestJSON-858074346 tempest-VolumesSnapshotsTestJSON-858074346-project] Calling method '.version_select at 0x7f6b1b602d30>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:22:40.806149 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-b490cecb-9b99-43b3-8796-87a9a2fd1c97 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '26a86351583d49d090b3db09f127d49f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5d5ec8450c0941d18d63558baab54737', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:22:40.823012 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-b490cecb-9b99-43b3-8796-87a9a2fd1c97 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] 10.222.0.52 "GET /compute/v2.1/servers/2ad09ab6-ef1f-47a2-bf44-6ae046cc8645" status: 200 len: 1245 microversion: 2.1 time: 0.189827 Aug 27 02:22:40.824672 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 15/30] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:22:40 2022] GET /compute/v2.1/servers/2ad09ab6-ef1f-47a2-bf44-6ae046cc8645 => generated 1245 bytes in 192 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:22:40.885926 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-6e280401-96f6-492e-8c44-9f4d8c941710 tempest-VolumesSnapshotsTestJSON-858074346 tempest-VolumesSnapshotsTestJSON-858074346-project] 10.222.0.52 "DELETE /compute/v2.1/os-snapshots/32064ba5-791a-467e-a835-6b782c963b35" status: 202 len: 0 microversion: 2.1 time: 0.207490 Aug 27 02:22:40.886739 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 16/31] 10.222.0.52 () {58 vars in 1254 bytes} [Sat Aug 27 02:22:40 2022] DELETE /compute/v2.1/os-snapshots/32064ba5-791a-467e-a835-6b782c963b35 => generated 0 bytes in 209 msecs (HTTP/1.1 202) 9 headers in 361 bytes (2 switches on core 0) Aug 27 02:22:40.896074 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-28854ea4-0325-40a9-bc3d-30cee9564e21 tempest-VolumesSnapshotsTestJSON-858074346 tempest-VolumesSnapshotsTestJSON-858074346-project] Calling method '.version_select at 0x7f6b1b5cd1f0>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:22:40.972190 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-28854ea4-0325-40a9-bc3d-30cee9564e21 tempest-VolumesSnapshotsTestJSON-858074346 tempest-VolumesSnapshotsTestJSON-858074346-project] 10.222.0.52 "GET /compute/v2.1/os-snapshots/32064ba5-791a-467e-a835-6b782c963b35" status: 200 len: 287 microversion: 2.1 time: 0.080978 Aug 27 02:22:40.973022 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 16/32] 10.222.0.52 () {58 vars in 1251 bytes} [Sat Aug 27 02:22:40 2022] GET /compute/v2.1/os-snapshots/32064ba5-791a-467e-a835-6b782c963b35 => generated 287 bytes in 82 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 27 02:22:41.241043 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-69b8fee0-922e-4c31-b83d-2c193167958f tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:22:41.241863 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-69b8fee0-922e-4c31-b83d-2c193167958f tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] [instance: fbef28bb-3cdf-4a27-8493-add7806af828] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:22:41.249990 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-69b8fee0-922e-4c31-b83d-2c193167958f tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:22:41.250679 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-69b8fee0-922e-4c31-b83d-2c193167958f tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:22:41.251287 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-69b8fee0-922e-4c31-b83d-2c193167958f tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:22:41.252177 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-69b8fee0-922e-4c31-b83d-2c193167958f tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:22:41.252873 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-69b8fee0-922e-4c31-b83d-2c193167958f tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:22:41.253425 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-69b8fee0-922e-4c31-b83d-2c193167958f tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:22:41.337732 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-69b8fee0-922e-4c31-b83d-2c193167958f tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '55640f750d1b49be9c3f79501a63c1ab', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7ef4b44a3f94d599eacce269b06594c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:22:41.347314 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-69b8fee0-922e-4c31-b83d-2c193167958f tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] 10.222.0.52 "GET /compute/v2.1/servers/fbef28bb-3cdf-4a27-8493-add7806af828" status: 200 len: 1266 microversion: 2.1 time: 0.111132 Aug 27 02:22:41.347911 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 17/33] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:22:41 2022] GET /compute/v2.1/servers/fbef28bb-3cdf-4a27-8493-add7806af828 => generated 1266 bytes in 112 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:22:41.622263 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-8f6e9a28-10bb-4c09-b8c9-f405ec67acbc tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Calling method '>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:22:41.623257 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-8f6e9a28-10bb-4c09-b8c9-f405ec67acbc tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] [instance: 05bed8e9-f530-418a-9819-4f65dabdcda0] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:22:41.629759 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-8f6e9a28-10bb-4c09-b8c9-f405ec67acbc tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:22:41.630326 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-8f6e9a28-10bb-4c09-b8c9-f405ec67acbc tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:22:41.631012 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-8f6e9a28-10bb-4c09-b8c9-f405ec67acbc tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:22:41.631972 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-8f6e9a28-10bb-4c09-b8c9-f405ec67acbc tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:22:41.632628 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-8f6e9a28-10bb-4c09-b8c9-f405ec67acbc tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:22:41.633138 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-8f6e9a28-10bb-4c09-b8c9-f405ec67acbc tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:22:41.708656 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-8f6e9a28-10bb-4c09-b8c9-f405ec67acbc tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd945d512ae034ec08838d0a3afde2429', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '880ac4003248441c809a76cfd1b221eb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:22:41.717515 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-8f6e9a28-10bb-4c09-b8c9-f405ec67acbc tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] 10.222.0.52 "GET /compute/v2.1/servers/05bed8e9-f530-418a-9819-4f65dabdcda0" status: 200 len: 1274 microversion: 2.1 time: 0.099065 Aug 27 02:22:41.717698 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 17/34] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:22:41 2022] GET /compute/v2.1/servers/05bed8e9-f530-418a-9819-4f65dabdcda0 => generated 1274 bytes in 101 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:22:41.859773 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-88d6245e-5fd4-4b91-84e2-bfe50bb57ed1 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:22:41.860393 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-88d6245e-5fd4-4b91-84e2-bfe50bb57ed1 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] [instance: 2ad09ab6-ef1f-47a2-bf44-6ae046cc8645] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:22:41.867789 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-88d6245e-5fd4-4b91-84e2-bfe50bb57ed1 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:22:41.868385 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-88d6245e-5fd4-4b91-84e2-bfe50bb57ed1 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:22:41.868875 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-88d6245e-5fd4-4b91-84e2-bfe50bb57ed1 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:22:41.869896 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-88d6245e-5fd4-4b91-84e2-bfe50bb57ed1 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:22:41.870355 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-88d6245e-5fd4-4b91-84e2-bfe50bb57ed1 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:22:41.870782 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-88d6245e-5fd4-4b91-84e2-bfe50bb57ed1 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:22:41.955851 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-88d6245e-5fd4-4b91-84e2-bfe50bb57ed1 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '26a86351583d49d090b3db09f127d49f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5d5ec8450c0941d18d63558baab54737', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:22:41.967404 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-88d6245e-5fd4-4b91-84e2-bfe50bb57ed1 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] 10.222.0.52 "GET /compute/v2.1/servers/2ad09ab6-ef1f-47a2-bf44-6ae046cc8645" status: 200 len: 1245 microversion: 2.1 time: 0.112796 Aug 27 02:22:41.968160 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 18/35] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:22:41 2022] GET /compute/v2.1/servers/2ad09ab6-ef1f-47a2-bf44-6ae046cc8645 => generated 1245 bytes in 115 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:22:41.987285 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-bd8888e1-ed31-4983-a9bd-f3746d6b69ec tempest-VolumesSnapshotsTestJSON-858074346 tempest-VolumesSnapshotsTestJSON-858074346-project] Calling method '.version_select at 0x7f6b1b5cd310>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:22:42.030074 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-bd8888e1-ed31-4983-a9bd-f3746d6b69ec tempest-VolumesSnapshotsTestJSON-858074346 tempest-VolumesSnapshotsTestJSON-858074346-project] 10.222.0.52 "GET /compute/v2.1/os-snapshots/32064ba5-791a-467e-a835-6b782c963b35" status: 200 len: 287 microversion: 2.1 time: 0.047238 Aug 27 02:22:42.035038 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 18/36] 10.222.0.52 () {58 vars in 1251 bytes} [Sat Aug 27 02:22:41 2022] GET /compute/v2.1/os-snapshots/32064ba5-791a-467e-a835-6b782c963b35 => generated 287 bytes in 53 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 27 02:22:42.376566 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-1b2e1283-181a-49ce-b399-fd82ebee32a7 tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:22:42.377478 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-1b2e1283-181a-49ce-b399-fd82ebee32a7 tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] [instance: fbef28bb-3cdf-4a27-8493-add7806af828] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:22:42.385069 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-1b2e1283-181a-49ce-b399-fd82ebee32a7 tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:22:42.386539 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-1b2e1283-181a-49ce-b399-fd82ebee32a7 tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:22:42.387161 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-1b2e1283-181a-49ce-b399-fd82ebee32a7 tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:22:42.388417 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-1b2e1283-181a-49ce-b399-fd82ebee32a7 tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:22:42.389275 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-1b2e1283-181a-49ce-b399-fd82ebee32a7 tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:22:42.389866 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-1b2e1283-181a-49ce-b399-fd82ebee32a7 tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:22:42.483725 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-1b2e1283-181a-49ce-b399-fd82ebee32a7 tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '55640f750d1b49be9c3f79501a63c1ab', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7ef4b44a3f94d599eacce269b06594c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:22:42.494521 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-1b2e1283-181a-49ce-b399-fd82ebee32a7 tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] 10.222.0.52 "GET /compute/v2.1/servers/fbef28bb-3cdf-4a27-8493-add7806af828" status: 200 len: 1266 microversion: 2.1 time: 0.124140 Aug 27 02:22:42.495114 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 19/37] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:22:42 2022] GET /compute/v2.1/servers/fbef28bb-3cdf-4a27-8493-add7806af828 => generated 1266 bytes in 125 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:22:42.741053 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-adfe4c98-3fc2-492a-9915-87e4b93e50a3 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Calling method '>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:22:42.741890 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-adfe4c98-3fc2-492a-9915-87e4b93e50a3 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] [instance: 05bed8e9-f530-418a-9819-4f65dabdcda0] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:22:42.751004 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-adfe4c98-3fc2-492a-9915-87e4b93e50a3 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:22:42.751494 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-adfe4c98-3fc2-492a-9915-87e4b93e50a3 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:22:42.751932 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-adfe4c98-3fc2-492a-9915-87e4b93e50a3 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:22:42.752840 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-adfe4c98-3fc2-492a-9915-87e4b93e50a3 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:22:42.753426 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-adfe4c98-3fc2-492a-9915-87e4b93e50a3 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:22:42.754016 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-adfe4c98-3fc2-492a-9915-87e4b93e50a3 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:22:42.848001 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-adfe4c98-3fc2-492a-9915-87e4b93e50a3 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd945d512ae034ec08838d0a3afde2429', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '880ac4003248441c809a76cfd1b221eb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:22:42.859283 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-adfe4c98-3fc2-492a-9915-87e4b93e50a3 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] 10.222.0.52 "GET /compute/v2.1/servers/05bed8e9-f530-418a-9819-4f65dabdcda0" status: 200 len: 1274 microversion: 2.1 time: 0.123983 Aug 27 02:22:42.860433 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 19/38] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:22:42 2022] GET /compute/v2.1/servers/05bed8e9-f530-418a-9819-4f65dabdcda0 => generated 1274 bytes in 126 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 27 02:22:42.990255 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-4e09fef8-4189-4418-b9d3-8f8c73ff484f tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:22:42.991256 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-4e09fef8-4189-4418-b9d3-8f8c73ff484f tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] [instance: 2ad09ab6-ef1f-47a2-bf44-6ae046cc8645] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:22:42.997851 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-4e09fef8-4189-4418-b9d3-8f8c73ff484f tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:22:42.998352 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-4e09fef8-4189-4418-b9d3-8f8c73ff484f tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:22:43.000346 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-4e09fef8-4189-4418-b9d3-8f8c73ff484f tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:22:43.000346 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-4e09fef8-4189-4418-b9d3-8f8c73ff484f tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:22:43.000346 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-4e09fef8-4189-4418-b9d3-8f8c73ff484f tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:22:43.001089 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-4e09fef8-4189-4418-b9d3-8f8c73ff484f tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:22:43.054730 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-6f165c24-59bb-4442-a52e-b2d8c024e658 tempest-VolumesSnapshotsTestJSON-858074346 tempest-VolumesSnapshotsTestJSON-858074346-project] Calling method '.version_select at 0x7f6b1b5cd1f0>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:22:43.093441 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-6f165c24-59bb-4442-a52e-b2d8c024e658 tempest-VolumesSnapshotsTestJSON-858074346 tempest-VolumesSnapshotsTestJSON-858074346-project] 10.222.0.52 "GET /compute/v2.1/os-snapshots/32064ba5-791a-467e-a835-6b782c963b35" status: 200 len: 287 microversion: 2.1 time: 0.043061 Aug 27 02:22:43.094335 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 20/39] 10.222.0.52 () {58 vars in 1251 bytes} [Sat Aug 27 02:22:43 2022] GET /compute/v2.1/os-snapshots/32064ba5-791a-467e-a835-6b782c963b35 => generated 287 bytes in 56 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 27 02:22:43.230825 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-4e09fef8-4189-4418-b9d3-8f8c73ff484f tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '26a86351583d49d090b3db09f127d49f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5d5ec8450c0941d18d63558baab54737', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:22:43.240591 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-4e09fef8-4189-4418-b9d3-8f8c73ff484f tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] 10.222.0.52 "GET /compute/v2.1/servers/2ad09ab6-ef1f-47a2-bf44-6ae046cc8645" status: 200 len: 1245 microversion: 2.1 time: 0.254377 Aug 27 02:22:43.242116 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 20/40] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:22:42 2022] GET /compute/v2.1/servers/2ad09ab6-ef1f-47a2-bf44-6ae046cc8645 => generated 1245 bytes in 257 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:22:43.524863 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-5453b079-e37f-48cd-a764-f57ff907fd5c tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Calling method '>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:22:43.524863 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-5453b079-e37f-48cd-a764-f57ff907fd5c tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] [instance: fbef28bb-3cdf-4a27-8493-add7806af828] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:22:43.530729 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-5453b079-e37f-48cd-a764-f57ff907fd5c tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:22:43.531235 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-5453b079-e37f-48cd-a764-f57ff907fd5c tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:22:43.531767 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-5453b079-e37f-48cd-a764-f57ff907fd5c tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:22:43.532479 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-5453b079-e37f-48cd-a764-f57ff907fd5c tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:22:43.532934 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-5453b079-e37f-48cd-a764-f57ff907fd5c tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:22:43.533396 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-5453b079-e37f-48cd-a764-f57ff907fd5c tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:22:43.606521 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-5453b079-e37f-48cd-a764-f57ff907fd5c tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '55640f750d1b49be9c3f79501a63c1ab', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7ef4b44a3f94d599eacce269b06594c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:22:43.623526 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-5453b079-e37f-48cd-a764-f57ff907fd5c tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] 10.222.0.52 "GET /compute/v2.1/servers/fbef28bb-3cdf-4a27-8493-add7806af828" status: 200 len: 1266 microversion: 2.1 time: 0.104585 Aug 27 02:22:43.624482 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 21/41] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:22:43 2022] GET /compute/v2.1/servers/fbef28bb-3cdf-4a27-8493-add7806af828 => generated 1266 bytes in 106 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:22:43.881805 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-c4afc545-e87a-4e92-8a0b-49f99931a019 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:22:43.882447 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-c4afc545-e87a-4e92-8a0b-49f99931a019 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] [instance: 05bed8e9-f530-418a-9819-4f65dabdcda0] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:22:43.888912 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-c4afc545-e87a-4e92-8a0b-49f99931a019 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:22:43.891207 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-c4afc545-e87a-4e92-8a0b-49f99931a019 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:22:43.891207 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-c4afc545-e87a-4e92-8a0b-49f99931a019 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:22:43.891207 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-c4afc545-e87a-4e92-8a0b-49f99931a019 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:22:43.891657 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-c4afc545-e87a-4e92-8a0b-49f99931a019 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:22:43.891657 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-c4afc545-e87a-4e92-8a0b-49f99931a019 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:22:43.993112 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-c4afc545-e87a-4e92-8a0b-49f99931a019 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd945d512ae034ec08838d0a3afde2429', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '880ac4003248441c809a76cfd1b221eb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:22:44.002815 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-c4afc545-e87a-4e92-8a0b-49f99931a019 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] 10.222.0.52 "GET /compute/v2.1/servers/05bed8e9-f530-418a-9819-4f65dabdcda0" status: 200 len: 1274 microversion: 2.1 time: 0.125370 Aug 27 02:22:44.003576 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 21/42] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:22:43 2022] GET /compute/v2.1/servers/05bed8e9-f530-418a-9819-4f65dabdcda0 => generated 1274 bytes in 127 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:22:44.114545 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-dcf31018-6b87-451f-aad4-56ebbd4fc9c7 tempest-VolumesSnapshotsTestJSON-858074346 tempest-VolumesSnapshotsTestJSON-858074346-project] Calling method '.version_select at 0x7f6b1b5e1af0>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:22:44.170168 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-dcf31018-6b87-451f-aad4-56ebbd4fc9c7 tempest-VolumesSnapshotsTestJSON-858074346 tempest-VolumesSnapshotsTestJSON-858074346-project] 10.222.0.52 "GET /compute/v2.1/os-snapshots/32064ba5-791a-467e-a835-6b782c963b35" status: 200 len: 287 microversion: 2.1 time: 0.064556 Aug 27 02:22:44.170168 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 22/43] 10.222.0.52 () {58 vars in 1251 bytes} [Sat Aug 27 02:22:44 2022] GET /compute/v2.1/os-snapshots/32064ba5-791a-467e-a835-6b782c963b35 => generated 287 bytes in 67 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 27 02:22:44.268975 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-d079d7b0-3cba-4302-b657-233afb9d0ccd tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:22:44.270173 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-d079d7b0-3cba-4302-b657-233afb9d0ccd tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] [instance: 2ad09ab6-ef1f-47a2-bf44-6ae046cc8645] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:22:44.309466 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-d079d7b0-3cba-4302-b657-233afb9d0ccd tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:22:44.311381 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-d079d7b0-3cba-4302-b657-233afb9d0ccd tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:22:44.314755 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-d079d7b0-3cba-4302-b657-233afb9d0ccd tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:22:44.319608 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-d079d7b0-3cba-4302-b657-233afb9d0ccd tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:22:44.324842 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-d079d7b0-3cba-4302-b657-233afb9d0ccd tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:22:44.325406 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-d079d7b0-3cba-4302-b657-233afb9d0ccd tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:22:44.487422 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-d079d7b0-3cba-4302-b657-233afb9d0ccd tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '26a86351583d49d090b3db09f127d49f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5d5ec8450c0941d18d63558baab54737', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:22:44.498518 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-d079d7b0-3cba-4302-b657-233afb9d0ccd tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] 10.222.0.52 "GET /compute/v2.1/servers/2ad09ab6-ef1f-47a2-bf44-6ae046cc8645" status: 200 len: 1237 microversion: 2.1 time: 0.236308 Aug 27 02:22:44.499112 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 22/44] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:22:44 2022] GET /compute/v2.1/servers/2ad09ab6-ef1f-47a2-bf44-6ae046cc8645 => generated 1237 bytes in 238 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:22:44.656800 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-606db8c9-e871-43c5-b01a-eeec3798602f tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Calling method '>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:22:44.657792 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-606db8c9-e871-43c5-b01a-eeec3798602f tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] [instance: fbef28bb-3cdf-4a27-8493-add7806af828] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:22:44.668987 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-606db8c9-e871-43c5-b01a-eeec3798602f tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:22:44.670047 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-606db8c9-e871-43c5-b01a-eeec3798602f tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:22:44.671227 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-606db8c9-e871-43c5-b01a-eeec3798602f tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:22:44.672643 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-606db8c9-e871-43c5-b01a-eeec3798602f tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:22:44.674062 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-606db8c9-e871-43c5-b01a-eeec3798602f tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:22:44.674818 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-606db8c9-e871-43c5-b01a-eeec3798602f tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:22:44.759267 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-606db8c9-e871-43c5-b01a-eeec3798602f tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '55640f750d1b49be9c3f79501a63c1ab', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7ef4b44a3f94d599eacce269b06594c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:22:44.767501 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-606db8c9-e871-43c5-b01a-eeec3798602f tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] 10.222.0.52 "GET /compute/v2.1/servers/fbef28bb-3cdf-4a27-8493-add7806af828" status: 200 len: 1266 microversion: 2.1 time: 0.119493 Aug 27 02:22:44.768088 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 23/45] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:22:44 2022] GET /compute/v2.1/servers/fbef28bb-3cdf-4a27-8493-add7806af828 => generated 1266 bytes in 121 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 27 02:22:45.029859 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-ee285dc7-13ce-4f81-9a2d-9ca2e9d528a3 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:22:45.030442 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-ee285dc7-13ce-4f81-9a2d-9ca2e9d528a3 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] [instance: 05bed8e9-f530-418a-9819-4f65dabdcda0] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:22:45.036838 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-ee285dc7-13ce-4f81-9a2d-9ca2e9d528a3 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:22:45.037366 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-ee285dc7-13ce-4f81-9a2d-9ca2e9d528a3 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:22:45.037769 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-ee285dc7-13ce-4f81-9a2d-9ca2e9d528a3 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:22:45.038672 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-ee285dc7-13ce-4f81-9a2d-9ca2e9d528a3 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:22:45.039151 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-ee285dc7-13ce-4f81-9a2d-9ca2e9d528a3 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:22:45.039594 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-ee285dc7-13ce-4f81-9a2d-9ca2e9d528a3 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:22:45.106198 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-ee285dc7-13ce-4f81-9a2d-9ca2e9d528a3 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd945d512ae034ec08838d0a3afde2429', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '880ac4003248441c809a76cfd1b221eb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:22:45.114343 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-ee285dc7-13ce-4f81-9a2d-9ca2e9d528a3 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] 10.222.0.52 "GET /compute/v2.1/servers/05bed8e9-f530-418a-9819-4f65dabdcda0" status: 200 len: 1274 microversion: 2.1 time: 0.088514 Aug 27 02:22:45.114903 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 23/46] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:22:45 2022] GET /compute/v2.1/servers/05bed8e9-f530-418a-9819-4f65dabdcda0 => generated 1274 bytes in 90 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:22:45.183856 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-0c7d8339-a81f-4a1a-bbe7-b3ce34b4694a tempest-VolumesSnapshotsTestJSON-858074346 tempest-VolumesSnapshotsTestJSON-858074346-project] Calling method '.version_select at 0x7f6b1b5ef280>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:22:45.217377 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-0c7d8339-a81f-4a1a-bbe7-b3ce34b4694a tempest-VolumesSnapshotsTestJSON-858074346 tempest-VolumesSnapshotsTestJSON-858074346-project] 10.222.0.52 "GET /compute/v2.1/os-snapshots/32064ba5-791a-467e-a835-6b782c963b35" status: 200 len: 287 microversion: 2.1 time: 0.037370 Aug 27 02:22:45.217946 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 24/47] 10.222.0.52 () {58 vars in 1251 bytes} [Sat Aug 27 02:22:45 2022] GET /compute/v2.1/os-snapshots/32064ba5-791a-467e-a835-6b782c963b35 => generated 287 bytes in 39 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 27 02:22:45.522976 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-3e88e462-9575-4583-a362-7e997a814105 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:22:45.523592 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-3e88e462-9575-4583-a362-7e997a814105 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] [instance: 2ad09ab6-ef1f-47a2-bf44-6ae046cc8645] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:22:45.530482 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-3e88e462-9575-4583-a362-7e997a814105 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:22:45.531432 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-3e88e462-9575-4583-a362-7e997a814105 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:22:45.532046 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-3e88e462-9575-4583-a362-7e997a814105 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:22:45.532856 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-3e88e462-9575-4583-a362-7e997a814105 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:22:45.533356 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-3e88e462-9575-4583-a362-7e997a814105 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:22:45.533789 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-3e88e462-9575-4583-a362-7e997a814105 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:22:45.614542 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-3e88e462-9575-4583-a362-7e997a814105 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '26a86351583d49d090b3db09f127d49f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5d5ec8450c0941d18d63558baab54737', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:22:45.622558 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-3e88e462-9575-4583-a362-7e997a814105 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] 10.222.0.52 "GET /compute/v2.1/servers/2ad09ab6-ef1f-47a2-bf44-6ae046cc8645" status: 200 len: 1293 microversion: 2.1 time: 0.103522 Aug 27 02:22:45.623153 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 24/48] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:22:45 2022] GET /compute/v2.1/servers/2ad09ab6-ef1f-47a2-bf44-6ae046cc8645 => generated 1293 bytes in 105 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:22:45.790692 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-5e62196f-8b68-4d9f-b046-a4efb6c5004c tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Calling method '>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:22:45.791307 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-5e62196f-8b68-4d9f-b046-a4efb6c5004c tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] [instance: fbef28bb-3cdf-4a27-8493-add7806af828] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:22:45.797768 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-5e62196f-8b68-4d9f-b046-a4efb6c5004c tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:22:45.798255 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-5e62196f-8b68-4d9f-b046-a4efb6c5004c tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:22:45.798686 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-5e62196f-8b68-4d9f-b046-a4efb6c5004c tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:22:45.801212 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-5e62196f-8b68-4d9f-b046-a4efb6c5004c tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:22:45.801798 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-5e62196f-8b68-4d9f-b046-a4efb6c5004c tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:22:45.802223 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-5e62196f-8b68-4d9f-b046-a4efb6c5004c tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:22:45.870284 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-5e62196f-8b68-4d9f-b046-a4efb6c5004c tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '55640f750d1b49be9c3f79501a63c1ab', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7ef4b44a3f94d599eacce269b06594c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:22:45.879843 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-5e62196f-8b68-4d9f-b046-a4efb6c5004c tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] 10.222.0.52 "GET /compute/v2.1/servers/fbef28bb-3cdf-4a27-8493-add7806af828" status: 200 len: 1266 microversion: 2.1 time: 0.093665 Aug 27 02:22:45.880397 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 25/49] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:22:45 2022] GET /compute/v2.1/servers/fbef28bb-3cdf-4a27-8493-add7806af828 => generated 1266 bytes in 95 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 27 02:22:46.137782 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-50168893-b430-4e30-8964-4202c73d2541 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:22:46.138376 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-50168893-b430-4e30-8964-4202c73d2541 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] [instance: 05bed8e9-f530-418a-9819-4f65dabdcda0] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:22:46.146151 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-50168893-b430-4e30-8964-4202c73d2541 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:22:46.146151 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-50168893-b430-4e30-8964-4202c73d2541 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:22:46.150074 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-50168893-b430-4e30-8964-4202c73d2541 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:22:46.150074 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-50168893-b430-4e30-8964-4202c73d2541 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:22:46.150074 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-50168893-b430-4e30-8964-4202c73d2541 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:22:46.150074 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-50168893-b430-4e30-8964-4202c73d2541 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:22:46.222061 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-50168893-b430-4e30-8964-4202c73d2541 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd945d512ae034ec08838d0a3afde2429', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '880ac4003248441c809a76cfd1b221eb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:22:46.231892 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-50168893-b430-4e30-8964-4202c73d2541 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] 10.222.0.52 "GET /compute/v2.1/servers/05bed8e9-f530-418a-9819-4f65dabdcda0" status: 200 len: 1274 microversion: 2.1 time: 0.099050 Aug 27 02:22:46.232982 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 25/50] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:22:46 2022] GET /compute/v2.1/servers/05bed8e9-f530-418a-9819-4f65dabdcda0 => generated 1274 bytes in 101 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:22:46.234379 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-3e468590-0145-4205-94ae-c52c34cdf352 tempest-VolumesSnapshotsTestJSON-858074346 tempest-VolumesSnapshotsTestJSON-858074346-project] Calling method '.version_select at 0x7f6b1b595ee0>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:22:46.275934 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-3e468590-0145-4205-94ae-c52c34cdf352 tempest-VolumesSnapshotsTestJSON-858074346 tempest-VolumesSnapshotsTestJSON-858074346-project] 10.222.0.52 "GET /compute/v2.1/os-snapshots/32064ba5-791a-467e-a835-6b782c963b35" status: 200 len: 287 microversion: 2.1 time: 0.048976 Aug 27 02:22:46.282781 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 26/51] 10.222.0.52 () {58 vars in 1251 bytes} [Sat Aug 27 02:22:46 2022] GET /compute/v2.1/os-snapshots/32064ba5-791a-467e-a835-6b782c963b35 => generated 287 bytes in 57 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 27 02:22:46.643820 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-88795c13-1fa7-4235-8c37-037a80675e16 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:22:46.644416 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-88795c13-1fa7-4235-8c37-037a80675e16 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] [instance: 2ad09ab6-ef1f-47a2-bf44-6ae046cc8645] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:22:46.651415 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-88795c13-1fa7-4235-8c37-037a80675e16 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:22:46.651906 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-88795c13-1fa7-4235-8c37-037a80675e16 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:22:46.652342 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-88795c13-1fa7-4235-8c37-037a80675e16 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:22:46.653077 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-88795c13-1fa7-4235-8c37-037a80675e16 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:22:46.653568 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-88795c13-1fa7-4235-8c37-037a80675e16 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:22:46.654143 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-88795c13-1fa7-4235-8c37-037a80675e16 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:22:46.720452 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-88795c13-1fa7-4235-8c37-037a80675e16 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '26a86351583d49d090b3db09f127d49f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5d5ec8450c0941d18d63558baab54737', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:22:46.728692 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-88795c13-1fa7-4235-8c37-037a80675e16 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] 10.222.0.52 "GET /compute/v2.1/servers/2ad09ab6-ef1f-47a2-bf44-6ae046cc8645" status: 200 len: 1293 microversion: 2.1 time: 0.088971 Aug 27 02:22:46.729291 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 26/52] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:22:46 2022] GET /compute/v2.1/servers/2ad09ab6-ef1f-47a2-bf44-6ae046cc8645 => generated 1293 bytes in 90 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:22:46.903690 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-4e04356c-425b-42e4-9859-bbc0bb66f401 tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Calling method '>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:22:46.904351 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-4e04356c-425b-42e4-9859-bbc0bb66f401 tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] [instance: fbef28bb-3cdf-4a27-8493-add7806af828] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:22:46.910893 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-4e04356c-425b-42e4-9859-bbc0bb66f401 tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:22:46.911410 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-4e04356c-425b-42e4-9859-bbc0bb66f401 tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:22:46.911827 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-4e04356c-425b-42e4-9859-bbc0bb66f401 tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:22:46.912733 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-4e04356c-425b-42e4-9859-bbc0bb66f401 tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:22:46.913231 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-4e04356c-425b-42e4-9859-bbc0bb66f401 tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:22:46.913665 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-4e04356c-425b-42e4-9859-bbc0bb66f401 tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:22:46.975873 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-4e04356c-425b-42e4-9859-bbc0bb66f401 tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '55640f750d1b49be9c3f79501a63c1ab', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7ef4b44a3f94d599eacce269b06594c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:22:46.983541 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-4e04356c-425b-42e4-9859-bbc0bb66f401 tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] 10.222.0.52 "GET /compute/v2.1/servers/fbef28bb-3cdf-4a27-8493-add7806af828" status: 200 len: 1266 microversion: 2.1 time: 0.083959 Aug 27 02:22:46.984135 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 27/53] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:22:46 2022] GET /compute/v2.1/servers/fbef28bb-3cdf-4a27-8493-add7806af828 => generated 1266 bytes in 85 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:22:47.267088 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-c617994a-ab34-4ec7-b861-04a60471bd2c tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:22:47.267645 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-c617994a-ab34-4ec7-b861-04a60471bd2c tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] [instance: 05bed8e9-f530-418a-9819-4f65dabdcda0] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:22:47.274089 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-c617994a-ab34-4ec7-b861-04a60471bd2c tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:22:47.274572 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-c617994a-ab34-4ec7-b861-04a60471bd2c tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:22:47.275004 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-c617994a-ab34-4ec7-b861-04a60471bd2c tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:22:47.275697 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-c617994a-ab34-4ec7-b861-04a60471bd2c tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:22:47.276147 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-c617994a-ab34-4ec7-b861-04a60471bd2c tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:22:47.276667 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-c617994a-ab34-4ec7-b861-04a60471bd2c tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:22:47.290327 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-a3c54574-1267-412e-a698-86a7da4db97d tempest-VolumesSnapshotsTestJSON-858074346 tempest-VolumesSnapshotsTestJSON-858074346-project] Calling method '.version_select at 0x7f6b1b5ef940>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:22:47.322346 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-a3c54574-1267-412e-a698-86a7da4db97d tempest-VolumesSnapshotsTestJSON-858074346 tempest-VolumesSnapshotsTestJSON-858074346-project] 10.222.0.52 "GET /compute/v2.1/os-snapshots/32064ba5-791a-467e-a835-6b782c963b35" status: 200 len: 287 microversion: 2.1 time: 0.036036 Aug 27 02:22:47.322915 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 28/54] 10.222.0.52 () {58 vars in 1251 bytes} [Sat Aug 27 02:22:47 2022] GET /compute/v2.1/os-snapshots/32064ba5-791a-467e-a835-6b782c963b35 => generated 287 bytes in 37 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 27 02:22:47.343255 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-c617994a-ab34-4ec7-b861-04a60471bd2c tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd945d512ae034ec08838d0a3afde2429', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '880ac4003248441c809a76cfd1b221eb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:22:47.351243 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-c617994a-ab34-4ec7-b861-04a60471bd2c tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] 10.222.0.52 "GET /compute/v2.1/servers/05bed8e9-f530-418a-9819-4f65dabdcda0" status: 200 len: 1274 microversion: 2.1 time: 0.088287 Aug 27 02:22:47.351808 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 27/55] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:22:47 2022] GET /compute/v2.1/servers/05bed8e9-f530-418a-9819-4f65dabdcda0 => generated 1274 bytes in 89 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:22:47.749761 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-d32fbb2c-9eb9-4a9b-b068-bb46e6e610ee tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Calling method '>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:22:47.750368 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-d32fbb2c-9eb9-4a9b-b068-bb46e6e610ee tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] [instance: 2ad09ab6-ef1f-47a2-bf44-6ae046cc8645] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:22:47.757564 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-d32fbb2c-9eb9-4a9b-b068-bb46e6e610ee tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:22:47.758276 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-d32fbb2c-9eb9-4a9b-b068-bb46e6e610ee tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:22:47.758836 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-d32fbb2c-9eb9-4a9b-b068-bb46e6e610ee tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:22:47.759541 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-d32fbb2c-9eb9-4a9b-b068-bb46e6e610ee tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:22:47.760245 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-d32fbb2c-9eb9-4a9b-b068-bb46e6e610ee tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:22:47.760763 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-d32fbb2c-9eb9-4a9b-b068-bb46e6e610ee tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:22:47.833833 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-d32fbb2c-9eb9-4a9b-b068-bb46e6e610ee tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '26a86351583d49d090b3db09f127d49f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5d5ec8450c0941d18d63558baab54737', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:22:47.841973 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-d32fbb2c-9eb9-4a9b-b068-bb46e6e610ee tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] 10.222.0.52 "GET /compute/v2.1/servers/2ad09ab6-ef1f-47a2-bf44-6ae046cc8645" status: 200 len: 1293 microversion: 2.1 time: 0.095939 Aug 27 02:22:47.842550 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 29/56] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:22:47 2022] GET /compute/v2.1/servers/2ad09ab6-ef1f-47a2-bf44-6ae046cc8645 => generated 1293 bytes in 97 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:22:48.006204 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-5c77b88d-69a6-4953-93ee-e6ae1582c888 tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:22:48.007195 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-5c77b88d-69a6-4953-93ee-e6ae1582c888 tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] [instance: fbef28bb-3cdf-4a27-8493-add7806af828] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:22:48.014697 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-5c77b88d-69a6-4953-93ee-e6ae1582c888 tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:22:48.015788 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-5c77b88d-69a6-4953-93ee-e6ae1582c888 tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:22:48.016471 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-5c77b88d-69a6-4953-93ee-e6ae1582c888 tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:22:48.017799 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-5c77b88d-69a6-4953-93ee-e6ae1582c888 tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:22:48.018765 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-5c77b88d-69a6-4953-93ee-e6ae1582c888 tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:22:48.019201 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-5c77b88d-69a6-4953-93ee-e6ae1582c888 tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:22:48.084106 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-5c77b88d-69a6-4953-93ee-e6ae1582c888 tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '55640f750d1b49be9c3f79501a63c1ab', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7ef4b44a3f94d599eacce269b06594c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:22:48.092220 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-5c77b88d-69a6-4953-93ee-e6ae1582c888 tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] 10.222.0.52 "GET /compute/v2.1/servers/fbef28bb-3cdf-4a27-8493-add7806af828" status: 200 len: 1266 microversion: 2.1 time: 0.091945 Aug 27 02:22:48.092784 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 28/57] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:22:47 2022] GET /compute/v2.1/servers/fbef28bb-3cdf-4a27-8493-add7806af828 => generated 1266 bytes in 93 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:22:48.335015 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG dbcounter [-] [120864] Writing DB stats nova_cell0:SELECT=2 {{(pid=120864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 27 02:22:48.342902 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-1fc0055a-e78a-4322-83e2-5c1171e39505 tempest-VolumesSnapshotsTestJSON-858074346 tempest-VolumesSnapshotsTestJSON-858074346-project] Calling method '.version_select at 0x7f6b1b5ef280>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:22:48.374956 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-69328bb4-fcda-4e70-bc6b-91cedc33122f tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:22:48.376092 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-69328bb4-fcda-4e70-bc6b-91cedc33122f tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] [instance: 05bed8e9-f530-418a-9819-4f65dabdcda0] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:22:48.384457 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-1fc0055a-e78a-4322-83e2-5c1171e39505 tempest-VolumesSnapshotsTestJSON-858074346 tempest-VolumesSnapshotsTestJSON-858074346-project] 10.222.0.52 "GET /compute/v2.1/os-snapshots/32064ba5-791a-467e-a835-6b782c963b35" status: 200 len: 287 microversion: 2.1 time: 0.050338 Aug 27 02:22:48.385029 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 30/58] 10.222.0.52 () {58 vars in 1251 bytes} [Sat Aug 27 02:22:48 2022] GET /compute/v2.1/os-snapshots/32064ba5-791a-467e-a835-6b782c963b35 => generated 287 bytes in 52 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 27 02:22:48.387400 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-69328bb4-fcda-4e70-bc6b-91cedc33122f tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:22:48.387891 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-69328bb4-fcda-4e70-bc6b-91cedc33122f tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:22:48.388323 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-69328bb4-fcda-4e70-bc6b-91cedc33122f tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:22:48.389023 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-69328bb4-fcda-4e70-bc6b-91cedc33122f tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:22:48.389532 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-69328bb4-fcda-4e70-bc6b-91cedc33122f tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:22:48.389961 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-69328bb4-fcda-4e70-bc6b-91cedc33122f tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:22:48.453505 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-69328bb4-fcda-4e70-bc6b-91cedc33122f tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd945d512ae034ec08838d0a3afde2429', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '880ac4003248441c809a76cfd1b221eb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:22:48.461826 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-69328bb4-fcda-4e70-bc6b-91cedc33122f tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] 10.222.0.52 "GET /compute/v2.1/servers/05bed8e9-f530-418a-9819-4f65dabdcda0" status: 200 len: 1274 microversion: 2.1 time: 0.093785 Aug 27 02:22:48.462420 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 29/59] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:22:48 2022] GET /compute/v2.1/servers/05bed8e9-f530-418a-9819-4f65dabdcda0 => generated 1274 bytes in 95 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:22:48.868278 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-8b652ba3-6cb0-4600-926c-c24e475b2c6b tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Calling method '>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:22:48.872313 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-8b652ba3-6cb0-4600-926c-c24e475b2c6b tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] [instance: 2ad09ab6-ef1f-47a2-bf44-6ae046cc8645] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:22:48.883062 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-8b652ba3-6cb0-4600-926c-c24e475b2c6b tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:22:48.883843 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-8b652ba3-6cb0-4600-926c-c24e475b2c6b tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:22:48.884516 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-8b652ba3-6cb0-4600-926c-c24e475b2c6b tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:22:48.886126 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-8b652ba3-6cb0-4600-926c-c24e475b2c6b tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:22:48.886846 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-8b652ba3-6cb0-4600-926c-c24e475b2c6b tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:22:48.888617 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-8b652ba3-6cb0-4600-926c-c24e475b2c6b tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:22:48.969699 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-8b652ba3-6cb0-4600-926c-c24e475b2c6b tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '26a86351583d49d090b3db09f127d49f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5d5ec8450c0941d18d63558baab54737', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:22:48.977699 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-8b652ba3-6cb0-4600-926c-c24e475b2c6b tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] 10.222.0.52 "GET /compute/v2.1/servers/2ad09ab6-ef1f-47a2-bf44-6ae046cc8645" status: 200 len: 1293 microversion: 2.1 time: 0.117672 Aug 27 02:22:48.978265 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 31/60] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:22:48 2022] GET /compute/v2.1/servers/2ad09ab6-ef1f-47a2-bf44-6ae046cc8645 => generated 1293 bytes in 119 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:22:49.114793 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-78294759-db83-43e0-816f-6ce9c8149cd1 tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:22:49.116353 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-78294759-db83-43e0-816f-6ce9c8149cd1 tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] [instance: fbef28bb-3cdf-4a27-8493-add7806af828] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:22:49.132471 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-78294759-db83-43e0-816f-6ce9c8149cd1 tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:22:49.153201 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-78294759-db83-43e0-816f-6ce9c8149cd1 tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:22:49.154013 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-78294759-db83-43e0-816f-6ce9c8149cd1 tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.021s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:22:49.156023 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-78294759-db83-43e0-816f-6ce9c8149cd1 tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:22:49.157375 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-78294759-db83-43e0-816f-6ce9c8149cd1 tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:22:49.157814 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-78294759-db83-43e0-816f-6ce9c8149cd1 tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:22:49.280994 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG dbcounter [-] [120863] Writing DB stats nova_cell0:SELECT=6 {{(pid=120863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 27 02:22:49.309471 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG dbcounter [-] [120863] Writing DB stats nova_cell0:SELECT=24 {{(pid=120863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 27 02:22:49.380104 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-78294759-db83-43e0-816f-6ce9c8149cd1 tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '55640f750d1b49be9c3f79501a63c1ab', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7ef4b44a3f94d599eacce269b06594c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:22:49.394883 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-78294759-db83-43e0-816f-6ce9c8149cd1 tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] 10.222.0.52 "GET /compute/v2.1/servers/fbef28bb-3cdf-4a27-8493-add7806af828" status: 200 len: 1274 microversion: 2.1 time: 0.284046 Aug 27 02:22:49.396014 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 30/61] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:22:49 2022] GET /compute/v2.1/servers/fbef28bb-3cdf-4a27-8493-add7806af828 => generated 1274 bytes in 286 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:22:49.398370 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-369376ba-5da3-48fc-949e-96a49f94270c tempest-VolumesSnapshotsTestJSON-858074346 tempest-VolumesSnapshotsTestJSON-858074346-project] Calling method '.version_select at 0x7f6b1b53a3a0>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:22:49.455469 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-369376ba-5da3-48fc-949e-96a49f94270c tempest-VolumesSnapshotsTestJSON-858074346 tempest-VolumesSnapshotsTestJSON-858074346-project] 10.222.0.52 "GET /compute/v2.1/os-snapshots/32064ba5-791a-467e-a835-6b782c963b35" status: 200 len: 287 microversion: 2.1 time: 0.061761 Aug 27 02:22:49.456032 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 32/62] 10.222.0.52 () {58 vars in 1251 bytes} [Sat Aug 27 02:22:49 2022] GET /compute/v2.1/os-snapshots/32064ba5-791a-467e-a835-6b782c963b35 => generated 287 bytes in 63 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 27 02:22:49.487638 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-90aa405a-4f0c-4083-856f-4ffc9f4d470d tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:22:49.488905 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-90aa405a-4f0c-4083-856f-4ffc9f4d470d tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] [instance: 05bed8e9-f530-418a-9819-4f65dabdcda0] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:22:49.495601 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-90aa405a-4f0c-4083-856f-4ffc9f4d470d tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:22:49.496086 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-90aa405a-4f0c-4083-856f-4ffc9f4d470d tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:22:49.496520 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-90aa405a-4f0c-4083-856f-4ffc9f4d470d tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:22:49.497262 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-90aa405a-4f0c-4083-856f-4ffc9f4d470d tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:22:49.497712 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-90aa405a-4f0c-4083-856f-4ffc9f4d470d tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:22:49.498126 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-90aa405a-4f0c-4083-856f-4ffc9f4d470d tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:22:49.531609 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG dbcounter [-] [120863] Writing DB stats nova_cell1:SELECT=100 {{(pid=120863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 27 02:22:49.589885 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-90aa405a-4f0c-4083-856f-4ffc9f4d470d tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd945d512ae034ec08838d0a3afde2429', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '880ac4003248441c809a76cfd1b221eb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:22:49.598566 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-90aa405a-4f0c-4083-856f-4ffc9f4d470d tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] 10.222.0.52 "GET /compute/v2.1/servers/05bed8e9-f530-418a-9819-4f65dabdcda0" status: 200 len: 1280 microversion: 2.1 time: 0.118878 Aug 27 02:22:49.599135 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 31/63] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:22:49 2022] GET /compute/v2.1/servers/05bed8e9-f530-418a-9819-4f65dabdcda0 => generated 1280 bytes in 120 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 27 02:22:50.004153 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-845c997f-f3c5-4d0e-b349-2ac235b388a6 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Calling method '>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:22:50.004800 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-845c997f-f3c5-4d0e-b349-2ac235b388a6 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] [instance: 2ad09ab6-ef1f-47a2-bf44-6ae046cc8645] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:22:50.007660 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG dbcounter [-] [120864] Writing DB stats nova_cell0:SELECT=14 {{(pid=120864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 27 02:22:50.013152 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-845c997f-f3c5-4d0e-b349-2ac235b388a6 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:22:50.013772 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-845c997f-f3c5-4d0e-b349-2ac235b388a6 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:22:50.014381 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-845c997f-f3c5-4d0e-b349-2ac235b388a6 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:22:50.015132 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-845c997f-f3c5-4d0e-b349-2ac235b388a6 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:22:50.015734 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-845c997f-f3c5-4d0e-b349-2ac235b388a6 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:22:50.016250 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-845c997f-f3c5-4d0e-b349-2ac235b388a6 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:22:50.254612 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-845c997f-f3c5-4d0e-b349-2ac235b388a6 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '26a86351583d49d090b3db09f127d49f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5d5ec8450c0941d18d63558baab54737', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:22:50.266942 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-845c997f-f3c5-4d0e-b349-2ac235b388a6 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] 10.222.0.52 "GET /compute/v2.1/servers/2ad09ab6-ef1f-47a2-bf44-6ae046cc8645" status: 200 len: 1293 microversion: 2.1 time: 0.266745 Aug 27 02:22:50.285344 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 33/64] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:22:49 2022] GET /compute/v2.1/servers/2ad09ab6-ef1f-47a2-bf44-6ae046cc8645 => generated 1293 bytes in 283 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:22:50.427963 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-fda27b5d-3279-4dbd-9538-bac429ce267a tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:22:50.430720 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-fda27b5d-3279-4dbd-9538-bac429ce267a tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] [instance: fbef28bb-3cdf-4a27-8493-add7806af828] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:22:50.441895 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-fda27b5d-3279-4dbd-9538-bac429ce267a tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:22:50.442728 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-fda27b5d-3279-4dbd-9538-bac429ce267a tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:22:50.444404 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-fda27b5d-3279-4dbd-9538-bac429ce267a tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:22:50.445582 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-fda27b5d-3279-4dbd-9538-bac429ce267a tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:22:50.446266 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-fda27b5d-3279-4dbd-9538-bac429ce267a tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:22:50.446977 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-fda27b5d-3279-4dbd-9538-bac429ce267a tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:22:50.476718 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-3af84d3b-a698-423e-99ac-e111b9d970ed tempest-VolumesSnapshotsTestJSON-858074346 tempest-VolumesSnapshotsTestJSON-858074346-project] Calling method '.version_select at 0x7f6b1b53ac10>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:22:50.545392 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.wsgi [None req-3af84d3b-a698-423e-99ac-e111b9d970ed tempest-VolumesSnapshotsTestJSON-858074346 tempest-VolumesSnapshotsTestJSON-858074346-project] HTTP exception thrown: Snapshot 32064ba5-791a-467e-a835-6b782c963b35 could not be found. Aug 27 02:22:50.550420 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-3af84d3b-a698-423e-99ac-e111b9d970ed tempest-VolumesSnapshotsTestJSON-858074346 tempest-VolumesSnapshotsTestJSON-858074346-project] Request method failure captured: Aug 27 02:22:50.550420 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: request: GET /compute/v2.1/os-snapshots/32064ba5-791a-467e-a835-6b782c963b35 HTTP/1.1 Aug 27 02:22:50.550420 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: Accept: application/json Aug 27 02:22:50.550420 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: Accept-Encoding: identity Aug 27 02:22:50.550420 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: Connection: close Aug 27 02:22:50.550420 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: Content-Length: 0 Aug 27 02:22:50.550420 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: Content-Type: application/json Aug 27 02:22:50.550420 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: Host: 10.222.0.52 Aug 27 02:22:50.550420 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: Openstack-System-Scope: None Aug 27 02:22:50.550420 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: User-Agent: python-urllib3/1.26.11 Aug 27 02:22:50.550420 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-Auth-Token: gAAAAABjCX_csUPZFtkQjjgGolcJcJnlQ8gJxXMTk5Xbw21nMQyHKm38BwFU_pKdDDWXJOCL4TLMKp79dew4emK88Eoii5dWFi2OtyC6g3tWmXfiEsuSW3Ps084apX7E0-mD6msylyvtEbiX3NFQsXkDvlgYx639Y15tUzMZ3Cil1HQhyiGKHLk Aug 27 02:22:50.550420 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-Domain-Id: None Aug 27 02:22:50.550420 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-Domain-Name: None Aug 27 02:22:50.550420 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-Identity-Status: Confirmed Aug 27 02:22:50.550420 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-Is-Admin-Project: True Aug 27 02:22:50.550420 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-Project-Domain-Id: default Aug 27 02:22:50.550420 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-Project-Domain-Name: Default Aug 27 02:22:50.550420 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-Project-Id: 5c41fc70b9774df08ee9b819edc12ba8 Aug 27 02:22:50.550420 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-Project-Name: tempest-VolumesSnapshotsTestJSON-858074346 Aug 27 02:22:50.550420 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-Role: reader,member Aug 27 02:22:50.550420 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-Roles: reader,member Aug 27 02:22:50.551954 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-Service-Catalog: [{"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.52:8080", "publicURL": "http://10.222.0.52:8080/v1/AUTH_5c41fc70b9774df08ee9b819edc12ba8"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52:9696/networking"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/compute/v2/5c41fc70b9774df08ee9b819edc12ba8"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/image"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/volume/v3/5c41fc70b9774df08ee9b819edc12ba8"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/identity"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/volume/v3/5c41fc70b9774df08ee9b819edc12ba8"}]}] Aug 27 02:22:50.551954 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-Tenant: tempest-VolumesSnapshotsTestJSON-858074346 Aug 27 02:22:50.551954 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-Tenant-Id: 5c41fc70b9774df08ee9b819edc12ba8 Aug 27 02:22:50.551954 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-Tenant-Name: tempest-VolumesSnapshotsTestJSON-858074346 Aug 27 02:22:50.551954 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-User: tempest-VolumesSnapshotsTestJSON-858074346-project Aug 27 02:22:50.551954 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-User-Domain-Id: default Aug 27 02:22:50.551954 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-User-Domain-Name: Default Aug 27 02:22:50.551954 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-User-Id: 7aa09b0963d04b588386f1c0a96e2128 Aug 27 02:22:50.551954 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-User-Name: tempest-VolumesSnapshotsTestJSON-858074346-project Aug 27 02:22:50.551954 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: method: Aug 27 02:22:50.551954 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: exception: Snapshot 32064ba5-791a-467e-a835-6b782c963b35 could not be found. Aug 27 02:22:50.551954 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: action_args: {'id': '32064ba5-791a-467e-a835-6b782c963b35'} Aug 27 02:22:50.551954 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Aug 27 02:22:50.553212 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 27 02:22:50.553212 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 464, in wrapper Aug 27 02:22:50.553212 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi res = method(self, ctx, snapshot_id, *args, **kwargs) Aug 27 02:22:50.553212 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 701, in get_snapshot Aug 27 02:22:50.553212 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi item = cinderclient(context).volume_snapshots.get(snapshot_id) Aug 27 02:22:50.553212 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volume_snapshots.py", line 160, in get Aug 27 02:22:50.553212 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi return self._get("/snapshots/%s" % snapshot_id, "snapshot") Aug 27 02:22:50.553212 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 297, in _get Aug 27 02:22:50.553212 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.get(url) Aug 27 02:22:50.553212 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 224, in get Aug 27 02:22:50.553212 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'GET', **kwargs) Aug 27 02:22:50.553212 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Aug 27 02:22:50.553212 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Aug 27 02:22:50.553212 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Aug 27 02:22:50.553212 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Aug 27 02:22:50.553212 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.NotFound: Snapshot 32064ba5-791a-467e-a835-6b782c963b35 could not be found. (HTTP 404) (Request-ID: req-5de8a38a-8a58-43f3-8339-0031701c186a) Aug 27 02:22:50.553212 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi Aug 27 02:22:50.553212 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 27 02:22:50.568765 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi Aug 27 02:22:50.568765 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 27 02:22:50.568765 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 617, in show Aug 27 02:22:50.568765 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi vol = self.volume_api.get_snapshot(context, id) Aug 27 02:22:50.568765 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Aug 27 02:22:50.568765 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Aug 27 02:22:50.568765 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 466, in wrapper Aug 27 02:22:50.568765 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi _reraise(exception.SnapshotNotFound(snapshot_id=snapshot_id)) Aug 27 02:22:50.568765 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise Aug 27 02:22:50.568765 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) Aug 27 02:22:50.568765 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 464, in wrapper Aug 27 02:22:50.568765 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi res = method(self, ctx, snapshot_id, *args, **kwargs) Aug 27 02:22:50.568765 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 701, in get_snapshot Aug 27 02:22:50.568765 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi item = cinderclient(context).volume_snapshots.get(snapshot_id) Aug 27 02:22:50.568765 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volume_snapshots.py", line 160, in get Aug 27 02:22:50.568765 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi return self._get("/snapshots/%s" % snapshot_id, "snapshot") Aug 27 02:22:50.568765 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 297, in _get Aug 27 02:22:50.568765 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.get(url) Aug 27 02:22:50.568765 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 224, in get Aug 27 02:22:50.574808 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'GET', **kwargs) Aug 27 02:22:50.574808 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Aug 27 02:22:50.574808 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Aug 27 02:22:50.574808 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Aug 27 02:22:50.574808 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Aug 27 02:22:50.574808 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi nova.exception.SnapshotNotFound: Snapshot 32064ba5-791a-467e-a835-6b782c963b35 could not be found. Aug 27 02:22:50.574808 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi Aug 27 02:22:50.574808 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 27 02:22:50.574808 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi Aug 27 02:22:50.574808 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 27 02:22:50.574808 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 27 02:22:50.574808 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 27 02:22:50.574808 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Aug 27 02:22:50.574808 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Aug 27 02:22:50.574808 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Aug 27 02:22:50.574808 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Aug 27 02:22:50.574808 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Aug 27 02:22:50.574808 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Aug 27 02:22:50.574808 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 619, in show Aug 27 02:22:50.574808 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Aug 27 02:22:50.577506 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Snapshot 32064ba5-791a-467e-a835-6b782c963b35 could not be found. Aug 27 02:22:50.577506 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi Aug 27 02:22:50.577506 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 27 02:22:50.577506 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi Aug 27 02:22:50.577506 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 27 02:22:50.577506 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 27 02:22:50.577506 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 27 02:22:50.577506 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Aug 27 02:22:50.577506 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Aug 27 02:22:50.577506 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Snapshot 32064ba5-791a-467e-a835-6b782c963b35 could not be found. Aug 27 02:22:50.577506 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi Aug 27 02:22:50.577506 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-3af84d3b-a698-423e-99ac-e111b9d970ed tempest-VolumesSnapshotsTestJSON-858074346 tempest-VolumesSnapshotsTestJSON-858074346-project] Returning 404 to user: Snapshot 32064ba5-791a-467e-a835-6b782c963b35 could not be found. {{(pid=120864) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Aug 27 02:22:50.577506 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-3af84d3b-a698-423e-99ac-e111b9d970ed tempest-VolumesSnapshotsTestJSON-858074346 tempest-VolumesSnapshotsTestJSON-858074346-project] 10.222.0.52 "GET /compute/v2.1/os-snapshots/32064ba5-791a-467e-a835-6b782c963b35" status: 404 len: 111 microversion: 2.1 time: 0.084269 Aug 27 02:22:50.577506 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 34/65] 10.222.0.52 () {58 vars in 1251 bytes} [Sat Aug 27 02:22:50 2022] GET /compute/v2.1/os-snapshots/32064ba5-791a-467e-a835-6b782c963b35 => generated 111 bytes in 91 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Aug 27 02:22:50.581757 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-a3eb901e-349e-461e-97cf-eb1011fd617f tempest-VolumesSnapshotsTestJSON-858074346 tempest-VolumesSnapshotsTestJSON-858074346-project] Calling method '.version_select at 0x7f6b1c9481f0>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:22:50.672580 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-fda27b5d-3279-4dbd-9538-bac429ce267a tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '55640f750d1b49be9c3f79501a63c1ab', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7ef4b44a3f94d599eacce269b06594c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:22:50.683510 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-fda27b5d-3279-4dbd-9538-bac429ce267a tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] 10.222.0.52 "GET /compute/v2.1/servers/fbef28bb-3cdf-4a27-8493-add7806af828" status: 200 len: 1272 microversion: 2.1 time: 0.268535 Aug 27 02:22:50.684217 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 32/66] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:22:50 2022] GET /compute/v2.1/servers/fbef28bb-3cdf-4a27-8493-add7806af828 => generated 1272 bytes in 270 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:22:50.706785 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-c411452f-67f9-4be9-8db1-df5581f067ce tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:22:50.707394 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-c411452f-67f9-4be9-8db1-df5581f067ce tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] [instance: 05bed8e9-f530-418a-9819-4f65dabdcda0] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:22:50.725355 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-c411452f-67f9-4be9-8db1-df5581f067ce tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:22:50.729836 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-c411452f-67f9-4be9-8db1-df5581f067ce tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:22:50.730517 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-c411452f-67f9-4be9-8db1-df5581f067ce tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:22:50.734716 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-c411452f-67f9-4be9-8db1-df5581f067ce tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:22:50.735267 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-c411452f-67f9-4be9-8db1-df5581f067ce tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:22:50.735773 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-c411452f-67f9-4be9-8db1-df5581f067ce tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:22:50.765198 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-a3eb901e-349e-461e-97cf-eb1011fd617f tempest-VolumesSnapshotsTestJSON-858074346 tempest-VolumesSnapshotsTestJSON-858074346-project] 10.222.0.52 "DELETE /compute/v2.1/os-volumes/41f7ec3a-cbc1-4f12-8eb3-b34b1f334cca" status: 202 len: 0 microversion: 2.1 time: 0.206542 Aug 27 02:22:50.765198 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 35/67] 10.222.0.52 () {58 vars in 1248 bytes} [Sat Aug 27 02:22:50 2022] DELETE /compute/v2.1/os-volumes/41f7ec3a-cbc1-4f12-8eb3-b34b1f334cca => generated 0 bytes in 208 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Aug 27 02:22:50.776673 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-a5af4850-3280-47e9-b999-8af6347d9c3d tempest-VolumesSnapshotsTestJSON-858074346 tempest-VolumesSnapshotsTestJSON-858074346-project] Calling method '.version_select at 0x7f6b1beb4160>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:22:50.884611 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-a5af4850-3280-47e9-b999-8af6347d9c3d tempest-VolumesSnapshotsTestJSON-858074346 tempest-VolumesSnapshotsTestJSON-858074346-project] 10.222.0.52 "GET /compute/v2.1/os-volumes/41f7ec3a-cbc1-4f12-8eb3-b34b1f334cca" status: 200 len: 344 microversion: 2.1 time: 0.113020 Aug 27 02:22:50.892294 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 36/68] 10.222.0.52 () {58 vars in 1245 bytes} [Sat Aug 27 02:22:50 2022] GET /compute/v2.1/os-volumes/41f7ec3a-cbc1-4f12-8eb3-b34b1f334cca => generated 344 bytes in 122 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 27 02:22:50.966169 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-c411452f-67f9-4be9-8db1-df5581f067ce tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd945d512ae034ec08838d0a3afde2429', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '880ac4003248441c809a76cfd1b221eb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:22:50.982929 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-c411452f-67f9-4be9-8db1-df5581f067ce tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] 10.222.0.52 "GET /compute/v2.1/servers/05bed8e9-f530-418a-9819-4f65dabdcda0" status: 200 len: 1280 microversion: 2.1 time: 0.279221 Aug 27 02:22:50.992350 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 33/69] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:22:50 2022] GET /compute/v2.1/servers/05bed8e9-f530-418a-9819-4f65dabdcda0 => generated 1280 bytes in 288 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:22:51.307744 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-0a88a984-15f2-48c2-a4d1-f1108f45a9e6 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Calling method '>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:22:51.309023 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-0a88a984-15f2-48c2-a4d1-f1108f45a9e6 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] [instance: 2ad09ab6-ef1f-47a2-bf44-6ae046cc8645] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:22:51.320700 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-0a88a984-15f2-48c2-a4d1-f1108f45a9e6 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:22:51.324776 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-0a88a984-15f2-48c2-a4d1-f1108f45a9e6 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:22:51.327172 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-0a88a984-15f2-48c2-a4d1-f1108f45a9e6 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:22:51.330074 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-0a88a984-15f2-48c2-a4d1-f1108f45a9e6 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:22:51.332754 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-0a88a984-15f2-48c2-a4d1-f1108f45a9e6 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:22:51.334524 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-0a88a984-15f2-48c2-a4d1-f1108f45a9e6 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:22:51.607005 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-0a88a984-15f2-48c2-a4d1-f1108f45a9e6 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '26a86351583d49d090b3db09f127d49f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5d5ec8450c0941d18d63558baab54737', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:22:51.639997 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-0a88a984-15f2-48c2-a4d1-f1108f45a9e6 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] 10.222.0.52 "GET /compute/v2.1/servers/2ad09ab6-ef1f-47a2-bf44-6ae046cc8645" status: 200 len: 1299 microversion: 2.1 time: 0.335808 Aug 27 02:22:51.646510 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 37/70] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:22:51 2022] GET /compute/v2.1/servers/2ad09ab6-ef1f-47a2-bf44-6ae046cc8645 => generated 1299 bytes in 346 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:22:51.715725 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-ef7af28f-3106-42cd-a712-61822fcad7b6 tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:22:51.716814 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-ef7af28f-3106-42cd-a712-61822fcad7b6 tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] [instance: fbef28bb-3cdf-4a27-8493-add7806af828] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:22:51.755789 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG dbcounter [-] [120863] Writing DB stats nova_api:SELECT=91,nova_api:INSERT=9 {{(pid=120863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 27 02:22:51.797956 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-ef7af28f-3106-42cd-a712-61822fcad7b6 tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:22:51.804456 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-ef7af28f-3106-42cd-a712-61822fcad7b6 tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.007s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:22:51.804968 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-ef7af28f-3106-42cd-a712-61822fcad7b6 tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:22:51.809571 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-ef7af28f-3106-42cd-a712-61822fcad7b6 tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:22:51.810059 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-ef7af28f-3106-42cd-a712-61822fcad7b6 tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:22:51.810493 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-ef7af28f-3106-42cd-a712-61822fcad7b6 tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:22:51.899293 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-7220b7be-c5a1-4f06-ad68-662035e689a4 tempest-VolumesSnapshotsTestJSON-858074346 tempest-VolumesSnapshotsTestJSON-858074346-project] Calling method '.version_select at 0x7f6b1b646040>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:22:52.028144 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.wsgi [None req-7220b7be-c5a1-4f06-ad68-662035e689a4 tempest-VolumesSnapshotsTestJSON-858074346 tempest-VolumesSnapshotsTestJSON-858074346-project] HTTP exception thrown: Volume 41f7ec3a-cbc1-4f12-8eb3-b34b1f334cca could not be found. Aug 27 02:22:52.033314 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-7220b7be-c5a1-4f06-ad68-662035e689a4 tempest-VolumesSnapshotsTestJSON-858074346 tempest-VolumesSnapshotsTestJSON-858074346-project] Request method failure captured: Aug 27 02:22:52.033314 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: request: GET /compute/v2.1/os-volumes/41f7ec3a-cbc1-4f12-8eb3-b34b1f334cca HTTP/1.1 Aug 27 02:22:52.033314 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: Accept: application/json Aug 27 02:22:52.033314 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: Accept-Encoding: identity Aug 27 02:22:52.033314 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: Connection: close Aug 27 02:22:52.033314 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: Content-Length: 0 Aug 27 02:22:52.033314 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: Content-Type: application/json Aug 27 02:22:52.033314 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: Host: 10.222.0.52 Aug 27 02:22:52.033314 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: Openstack-System-Scope: None Aug 27 02:22:52.033314 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: User-Agent: python-urllib3/1.26.11 Aug 27 02:22:52.033314 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-Auth-Token: gAAAAABjCX_csUPZFtkQjjgGolcJcJnlQ8gJxXMTk5Xbw21nMQyHKm38BwFU_pKdDDWXJOCL4TLMKp79dew4emK88Eoii5dWFi2OtyC6g3tWmXfiEsuSW3Ps084apX7E0-mD6msylyvtEbiX3NFQsXkDvlgYx639Y15tUzMZ3Cil1HQhyiGKHLk Aug 27 02:22:52.033314 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-Domain-Id: None Aug 27 02:22:52.033314 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-Domain-Name: None Aug 27 02:22:52.033314 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-Identity-Status: Confirmed Aug 27 02:22:52.033314 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-Is-Admin-Project: True Aug 27 02:22:52.033314 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-Project-Domain-Id: default Aug 27 02:22:52.033314 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-Project-Domain-Name: Default Aug 27 02:22:52.033314 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-Project-Id: 5c41fc70b9774df08ee9b819edc12ba8 Aug 27 02:22:52.033314 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-Project-Name: tempest-VolumesSnapshotsTestJSON-858074346 Aug 27 02:22:52.033314 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-Role: reader,member Aug 27 02:22:52.033314 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-Roles: reader,member Aug 27 02:22:52.034660 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-Service-Catalog: [{"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.52:8080", "publicURL": "http://10.222.0.52:8080/v1/AUTH_5c41fc70b9774df08ee9b819edc12ba8"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52:9696/networking"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/compute/v2/5c41fc70b9774df08ee9b819edc12ba8"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/image"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/volume/v3/5c41fc70b9774df08ee9b819edc12ba8"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/identity"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/volume/v3/5c41fc70b9774df08ee9b819edc12ba8"}]}] Aug 27 02:22:52.034660 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-Tenant: tempest-VolumesSnapshotsTestJSON-858074346 Aug 27 02:22:52.034660 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-Tenant-Id: 5c41fc70b9774df08ee9b819edc12ba8 Aug 27 02:22:52.034660 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-Tenant-Name: tempest-VolumesSnapshotsTestJSON-858074346 Aug 27 02:22:52.034660 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-User: tempest-VolumesSnapshotsTestJSON-858074346-project Aug 27 02:22:52.034660 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-User-Domain-Id: default Aug 27 02:22:52.034660 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-User-Domain-Name: Default Aug 27 02:22:52.034660 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-User-Id: 7aa09b0963d04b588386f1c0a96e2128 Aug 27 02:22:52.034660 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-User-Name: tempest-VolumesSnapshotsTestJSON-858074346-project Aug 27 02:22:52.034660 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: method: Aug 27 02:22:52.034660 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: exception: Volume 41f7ec3a-cbc1-4f12-8eb3-b34b1f334cca could not be found. Aug 27 02:22:52.034660 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: action_args: {'id': '41f7ec3a-cbc1-4f12-8eb3-b34b1f334cca'} Aug 27 02:22:52.034660 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Aug 27 02:22:52.035481 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 27 02:22:52.035481 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Aug 27 02:22:52.035481 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Aug 27 02:22:52.035481 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 502, in get Aug 27 02:22:52.035481 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi item = cinderclient( Aug 27 02:22:52.035481 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 202, in get Aug 27 02:22:52.035481 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi return self._get("/volumes/%s" % volume_id, "volume") Aug 27 02:22:52.035481 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 297, in _get Aug 27 02:22:52.035481 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.get(url) Aug 27 02:22:52.035481 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 224, in get Aug 27 02:22:52.035481 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'GET', **kwargs) Aug 27 02:22:52.035481 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Aug 27 02:22:52.035481 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Aug 27 02:22:52.035481 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Aug 27 02:22:52.035481 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Aug 27 02:22:52.035481 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.NotFound: Volume 41f7ec3a-cbc1-4f12-8eb3-b34b1f334cca could not be found. (HTTP 404) (Request-ID: req-c987c3e2-9609-4b15-a75b-b2c93d5b3f85) Aug 27 02:22:52.035481 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi Aug 27 02:22:52.035481 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 27 02:22:52.035481 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi Aug 27 02:22:52.037501 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 27 02:22:52.037501 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 120, in show Aug 27 02:22:52.037501 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi vol = self.volume_api.get(context, id) Aug 27 02:22:52.037501 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Aug 27 02:22:52.037501 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Aug 27 02:22:52.037501 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 435, in wrapper Aug 27 02:22:52.037501 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi _reraise(exception.VolumeNotFound(volume_id=volume_id)) Aug 27 02:22:52.037501 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise Aug 27 02:22:52.037501 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) Aug 27 02:22:52.037501 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Aug 27 02:22:52.037501 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Aug 27 02:22:52.037501 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 502, in get Aug 27 02:22:52.037501 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi item = cinderclient( Aug 27 02:22:52.037501 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 202, in get Aug 27 02:22:52.037501 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi return self._get("/volumes/%s" % volume_id, "volume") Aug 27 02:22:52.037501 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 297, in _get Aug 27 02:22:52.037501 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.get(url) Aug 27 02:22:52.037501 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 224, in get Aug 27 02:22:52.037501 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'GET', **kwargs) Aug 27 02:22:52.038797 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Aug 27 02:22:52.038797 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Aug 27 02:22:52.038797 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Aug 27 02:22:52.038797 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Aug 27 02:22:52.038797 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi nova.exception.VolumeNotFound: Volume 41f7ec3a-cbc1-4f12-8eb3-b34b1f334cca could not be found. Aug 27 02:22:52.038797 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi Aug 27 02:22:52.038797 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 27 02:22:52.038797 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi Aug 27 02:22:52.038797 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 27 02:22:52.038797 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 27 02:22:52.038797 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 27 02:22:52.038797 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Aug 27 02:22:52.038797 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Aug 27 02:22:52.038797 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Aug 27 02:22:52.038797 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Aug 27 02:22:52.038797 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Aug 27 02:22:52.038797 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Aug 27 02:22:52.038797 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 122, in show Aug 27 02:22:52.038797 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Aug 27 02:22:52.039796 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Volume 41f7ec3a-cbc1-4f12-8eb3-b34b1f334cca could not be found. Aug 27 02:22:52.039796 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi Aug 27 02:22:52.039796 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 27 02:22:52.039796 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi Aug 27 02:22:52.039796 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 27 02:22:52.039796 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 27 02:22:52.039796 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 27 02:22:52.039796 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Aug 27 02:22:52.039796 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Aug 27 02:22:52.039796 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Volume 41f7ec3a-cbc1-4f12-8eb3-b34b1f334cca could not be found. Aug 27 02:22:52.039796 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi Aug 27 02:22:52.042016 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-7220b7be-c5a1-4f06-ad68-662035e689a4 tempest-VolumesSnapshotsTestJSON-858074346 tempest-VolumesSnapshotsTestJSON-858074346-project] Returning 404 to user: Volume 41f7ec3a-cbc1-4f12-8eb3-b34b1f334cca could not be found. {{(pid=120864) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Aug 27 02:22:52.043560 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-7220b7be-c5a1-4f06-ad68-662035e689a4 tempest-VolumesSnapshotsTestJSON-858074346 tempest-VolumesSnapshotsTestJSON-858074346-project] 10.222.0.52 "GET /compute/v2.1/os-volumes/41f7ec3a-cbc1-4f12-8eb3-b34b1f334cca" status: 404 len: 109 microversion: 2.1 time: 0.148309 Aug 27 02:22:52.046105 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 38/71] 10.222.0.52 () {58 vars in 1245 bytes} [Sat Aug 27 02:22:51 2022] GET /compute/v2.1/os-volumes/41f7ec3a-cbc1-4f12-8eb3-b34b1f334cca => generated 109 bytes in 151 msecs (HTTP/1.1 404) 9 headers in 379 bytes (2 switches on core 0) Aug 27 02:22:52.060731 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-e36b4cfa-1c9d-4b97-bc5a-446e77276238 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Calling method '>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:22:52.065140 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-e36b4cfa-1c9d-4b97-bc5a-446e77276238 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] [instance: 05bed8e9-f530-418a-9819-4f65dabdcda0] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:22:52.088350 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-e36b4cfa-1c9d-4b97-bc5a-446e77276238 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:22:52.092774 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-e36b4cfa-1c9d-4b97-bc5a-446e77276238 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:22:52.094688 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-e36b4cfa-1c9d-4b97-bc5a-446e77276238 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:22:52.097485 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-e36b4cfa-1c9d-4b97-bc5a-446e77276238 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:22:52.098821 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-e36b4cfa-1c9d-4b97-bc5a-446e77276238 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:22:52.101921 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-e36b4cfa-1c9d-4b97-bc5a-446e77276238 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:22:52.273898 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-ef7af28f-3106-42cd-a712-61822fcad7b6 tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '55640f750d1b49be9c3f79501a63c1ab', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7ef4b44a3f94d599eacce269b06594c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:22:52.338083 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-ef7af28f-3106-42cd-a712-61822fcad7b6 tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] 10.222.0.52 "GET /compute/v2.1/servers/fbef28bb-3cdf-4a27-8493-add7806af828" status: 200 len: 1272 microversion: 2.1 time: 0.631299 Aug 27 02:22:52.362739 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 34/72] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:22:51 2022] GET /compute/v2.1/servers/fbef28bb-3cdf-4a27-8493-add7806af828 => generated 1272 bytes in 657 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:22:52.380090 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-e4aaff55-efdb-497c-895f-1b8311022c92 tempest-VolumesSnapshotsTestJSON-858074346 tempest-VolumesSnapshotsTestJSON-858074346-project] Calling method '.version_select at 0x7f6b1b5cdaf0>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:22:52.634695 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.wsgi [None req-e4aaff55-efdb-497c-895f-1b8311022c92 tempest-VolumesSnapshotsTestJSON-858074346 tempest-VolumesSnapshotsTestJSON-858074346-project] HTTP exception thrown: Volume 41f7ec3a-cbc1-4f12-8eb3-b34b1f334cca could not be found. Aug 27 02:22:52.640631 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-e4aaff55-efdb-497c-895f-1b8311022c92 tempest-VolumesSnapshotsTestJSON-858074346 tempest-VolumesSnapshotsTestJSON-858074346-project] Request method failure captured: Aug 27 02:22:52.640631 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: request: DELETE /compute/v2.1/os-volumes/41f7ec3a-cbc1-4f12-8eb3-b34b1f334cca HTTP/1.1 Aug 27 02:22:52.640631 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: Accept: application/json Aug 27 02:22:52.640631 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: Accept-Encoding: identity Aug 27 02:22:52.640631 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: Connection: close Aug 27 02:22:52.640631 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: Content-Length: 0 Aug 27 02:22:52.640631 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: Content-Type: application/json Aug 27 02:22:52.640631 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: Host: 10.222.0.52 Aug 27 02:22:52.640631 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: Openstack-System-Scope: None Aug 27 02:22:52.640631 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: User-Agent: python-urllib3/1.26.11 Aug 27 02:22:52.640631 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-Auth-Token: gAAAAABjCX_csUPZFtkQjjgGolcJcJnlQ8gJxXMTk5Xbw21nMQyHKm38BwFU_pKdDDWXJOCL4TLMKp79dew4emK88Eoii5dWFi2OtyC6g3tWmXfiEsuSW3Ps084apX7E0-mD6msylyvtEbiX3NFQsXkDvlgYx639Y15tUzMZ3Cil1HQhyiGKHLk Aug 27 02:22:52.640631 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-Domain-Id: None Aug 27 02:22:52.640631 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-Domain-Name: None Aug 27 02:22:52.640631 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-Identity-Status: Confirmed Aug 27 02:22:52.640631 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-Is-Admin-Project: True Aug 27 02:22:52.640631 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-Project-Domain-Id: default Aug 27 02:22:52.640631 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-Project-Domain-Name: Default Aug 27 02:22:52.640631 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-Project-Id: 5c41fc70b9774df08ee9b819edc12ba8 Aug 27 02:22:52.640631 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-Project-Name: tempest-VolumesSnapshotsTestJSON-858074346 Aug 27 02:22:52.640631 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-Role: reader,member Aug 27 02:22:52.640631 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-Roles: reader,member Aug 27 02:22:52.641971 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-Service-Catalog: [{"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.52:8080", "publicURL": "http://10.222.0.52:8080/v1/AUTH_5c41fc70b9774df08ee9b819edc12ba8"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52:9696/networking"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/compute/v2/5c41fc70b9774df08ee9b819edc12ba8"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/image"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/volume/v3/5c41fc70b9774df08ee9b819edc12ba8"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/identity"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/volume/v3/5c41fc70b9774df08ee9b819edc12ba8"}]}] Aug 27 02:22:52.641971 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-Tenant: tempest-VolumesSnapshotsTestJSON-858074346 Aug 27 02:22:52.641971 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-Tenant-Id: 5c41fc70b9774df08ee9b819edc12ba8 Aug 27 02:22:52.641971 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-Tenant-Name: tempest-VolumesSnapshotsTestJSON-858074346 Aug 27 02:22:52.641971 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-User: tempest-VolumesSnapshotsTestJSON-858074346-project Aug 27 02:22:52.641971 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-User-Domain-Id: default Aug 27 02:22:52.641971 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-User-Domain-Name: Default Aug 27 02:22:52.641971 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-User-Id: 7aa09b0963d04b588386f1c0a96e2128 Aug 27 02:22:52.641971 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-User-Name: tempest-VolumesSnapshotsTestJSON-858074346-project Aug 27 02:22:52.641971 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: method: Aug 27 02:22:52.641971 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: exception: Volume 41f7ec3a-cbc1-4f12-8eb3-b34b1f334cca could not be found. Aug 27 02:22:52.641971 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: action_args: {'id': '41f7ec3a-cbc1-4f12-8eb3-b34b1f334cca'} Aug 27 02:22:52.641971 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Aug 27 02:22:52.642824 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 27 02:22:52.642824 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Aug 27 02:22:52.642824 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Aug 27 02:22:52.642824 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 676, in delete Aug 27 02:22:52.642824 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.delete(volume_id) Aug 27 02:22:52.642824 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 235, in delete Aug 27 02:22:52.642824 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi return self._delete(loc) Aug 27 02:22:52.642824 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 313, in _delete Aug 27 02:22:52.642824 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.delete(url) Aug 27 02:22:52.642824 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 233, in delete Aug 27 02:22:52.642824 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'DELETE', **kwargs) Aug 27 02:22:52.642824 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Aug 27 02:22:52.642824 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Aug 27 02:22:52.642824 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Aug 27 02:22:52.642824 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Aug 27 02:22:52.642824 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.NotFound: Volume 41f7ec3a-cbc1-4f12-8eb3-b34b1f334cca could not be found. (HTTP 404) (Request-ID: req-58166db6-bf52-4b54-9b03-6b4f34c3d4e9) Aug 27 02:22:52.642824 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi Aug 27 02:22:52.642824 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 27 02:22:52.642824 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi Aug 27 02:22:52.643788 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 27 02:22:52.643788 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 136, in delete Aug 27 02:22:52.643788 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi self.volume_api.delete(context, id) Aug 27 02:22:52.643788 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Aug 27 02:22:52.643788 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Aug 27 02:22:52.643788 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 435, in wrapper Aug 27 02:22:52.643788 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi _reraise(exception.VolumeNotFound(volume_id=volume_id)) Aug 27 02:22:52.643788 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise Aug 27 02:22:52.643788 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) Aug 27 02:22:52.643788 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Aug 27 02:22:52.643788 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Aug 27 02:22:52.643788 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 676, in delete Aug 27 02:22:52.643788 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.delete(volume_id) Aug 27 02:22:52.643788 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 235, in delete Aug 27 02:22:52.643788 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi return self._delete(loc) Aug 27 02:22:52.643788 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 313, in _delete Aug 27 02:22:52.643788 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.delete(url) Aug 27 02:22:52.643788 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 233, in delete Aug 27 02:22:52.643788 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'DELETE', **kwargs) Aug 27 02:22:52.644829 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Aug 27 02:22:52.644829 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Aug 27 02:22:52.644829 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Aug 27 02:22:52.644829 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Aug 27 02:22:52.644829 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi nova.exception.VolumeNotFound: Volume 41f7ec3a-cbc1-4f12-8eb3-b34b1f334cca could not be found. Aug 27 02:22:52.644829 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi Aug 27 02:22:52.644829 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 27 02:22:52.644829 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi Aug 27 02:22:52.644829 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 27 02:22:52.644829 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 27 02:22:52.644829 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 27 02:22:52.644829 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Aug 27 02:22:52.644829 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Aug 27 02:22:52.644829 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Aug 27 02:22:52.644829 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Aug 27 02:22:52.644829 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Aug 27 02:22:52.644829 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Aug 27 02:22:52.644829 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 140, in delete Aug 27 02:22:52.644829 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Aug 27 02:22:52.645739 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Volume 41f7ec3a-cbc1-4f12-8eb3-b34b1f334cca could not be found. Aug 27 02:22:52.645739 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi Aug 27 02:22:52.645739 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 27 02:22:52.645739 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi Aug 27 02:22:52.645739 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 27 02:22:52.645739 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 27 02:22:52.645739 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 27 02:22:52.645739 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Aug 27 02:22:52.645739 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Aug 27 02:22:52.645739 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Volume 41f7ec3a-cbc1-4f12-8eb3-b34b1f334cca could not be found. Aug 27 02:22:52.645739 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi Aug 27 02:22:52.645739 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-e4aaff55-efdb-497c-895f-1b8311022c92 tempest-VolumesSnapshotsTestJSON-858074346 tempest-VolumesSnapshotsTestJSON-858074346-project] Returning 404 to user: Volume 41f7ec3a-cbc1-4f12-8eb3-b34b1f334cca could not be found. {{(pid=120863) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Aug 27 02:22:52.646381 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-e4aaff55-efdb-497c-895f-1b8311022c92 tempest-VolumesSnapshotsTestJSON-858074346 tempest-VolumesSnapshotsTestJSON-858074346-project] 10.222.0.52 "DELETE /compute/v2.1/os-volumes/41f7ec3a-cbc1-4f12-8eb3-b34b1f334cca" status: 404 len: 109 microversion: 2.1 time: 0.281305 Aug 27 02:22:52.647661 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 35/73] 10.222.0.52 () {58 vars in 1248 bytes} [Sat Aug 27 02:22:52 2022] DELETE /compute/v2.1/os-volumes/41f7ec3a-cbc1-4f12-8eb3-b34b1f334cca => generated 109 bytes in 284 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Aug 27 02:22:52.664117 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-c2597eb9-fb7e-45b0-ac29-c6451c6ddbe3 tempest-VolumesSnapshotsTestJSON-858074346 tempest-VolumesSnapshotsTestJSON-858074346-project] Calling method '.version_select at 0x7f6b1b67cca0>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:22:53.050711 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.wsgi [None req-c2597eb9-fb7e-45b0-ac29-c6451c6ddbe3 tempest-VolumesSnapshotsTestJSON-858074346 tempest-VolumesSnapshotsTestJSON-858074346-project] HTTP exception thrown: Volume 41f7ec3a-cbc1-4f12-8eb3-b34b1f334cca could not be found. Aug 27 02:22:53.053348 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-c2597eb9-fb7e-45b0-ac29-c6451c6ddbe3 tempest-VolumesSnapshotsTestJSON-858074346 tempest-VolumesSnapshotsTestJSON-858074346-project] Request method failure captured: Aug 27 02:22:53.053348 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: request: GET /compute/v2.1/os-volumes/41f7ec3a-cbc1-4f12-8eb3-b34b1f334cca HTTP/1.1 Aug 27 02:22:53.053348 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: Accept: application/json Aug 27 02:22:53.053348 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: Accept-Encoding: identity Aug 27 02:22:53.053348 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: Connection: close Aug 27 02:22:53.053348 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: Content-Length: 0 Aug 27 02:22:53.053348 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: Content-Type: application/json Aug 27 02:22:53.053348 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: Host: 10.222.0.52 Aug 27 02:22:53.053348 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: Openstack-System-Scope: None Aug 27 02:22:53.053348 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: User-Agent: python-urllib3/1.26.11 Aug 27 02:22:53.053348 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-Auth-Token: gAAAAABjCX_csUPZFtkQjjgGolcJcJnlQ8gJxXMTk5Xbw21nMQyHKm38BwFU_pKdDDWXJOCL4TLMKp79dew4emK88Eoii5dWFi2OtyC6g3tWmXfiEsuSW3Ps084apX7E0-mD6msylyvtEbiX3NFQsXkDvlgYx639Y15tUzMZ3Cil1HQhyiGKHLk Aug 27 02:22:53.053348 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-Domain-Id: None Aug 27 02:22:53.053348 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-Domain-Name: None Aug 27 02:22:53.053348 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-Identity-Status: Confirmed Aug 27 02:22:53.053348 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-Is-Admin-Project: True Aug 27 02:22:53.053348 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-Project-Domain-Id: default Aug 27 02:22:53.053348 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-Project-Domain-Name: Default Aug 27 02:22:53.053348 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-Project-Id: 5c41fc70b9774df08ee9b819edc12ba8 Aug 27 02:22:53.053348 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-Project-Name: tempest-VolumesSnapshotsTestJSON-858074346 Aug 27 02:22:53.053348 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-Role: reader,member Aug 27 02:22:53.053348 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-Roles: reader,member Aug 27 02:22:53.055361 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-Service-Catalog: [{"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.52:8080", "publicURL": "http://10.222.0.52:8080/v1/AUTH_5c41fc70b9774df08ee9b819edc12ba8"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52:9696/networking"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/compute/v2/5c41fc70b9774df08ee9b819edc12ba8"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/image"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/volume/v3/5c41fc70b9774df08ee9b819edc12ba8"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/identity"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/volume/v3/5c41fc70b9774df08ee9b819edc12ba8"}]}] Aug 27 02:22:53.055361 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-Tenant: tempest-VolumesSnapshotsTestJSON-858074346 Aug 27 02:22:53.055361 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-Tenant-Id: 5c41fc70b9774df08ee9b819edc12ba8 Aug 27 02:22:53.055361 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-Tenant-Name: tempest-VolumesSnapshotsTestJSON-858074346 Aug 27 02:22:53.055361 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-User: tempest-VolumesSnapshotsTestJSON-858074346-project Aug 27 02:22:53.055361 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-User-Domain-Id: default Aug 27 02:22:53.055361 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-User-Domain-Name: Default Aug 27 02:22:53.055361 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-User-Id: 7aa09b0963d04b588386f1c0a96e2128 Aug 27 02:22:53.055361 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-User-Name: tempest-VolumesSnapshotsTestJSON-858074346-project Aug 27 02:22:53.055361 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: method: Aug 27 02:22:53.055361 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: exception: Volume 41f7ec3a-cbc1-4f12-8eb3-b34b1f334cca could not be found. Aug 27 02:22:53.055361 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: action_args: {'id': '41f7ec3a-cbc1-4f12-8eb3-b34b1f334cca'} Aug 27 02:22:53.055361 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Aug 27 02:22:53.056065 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 27 02:22:53.056065 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Aug 27 02:22:53.056065 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Aug 27 02:22:53.056065 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 502, in get Aug 27 02:22:53.056065 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi item = cinderclient( Aug 27 02:22:53.056065 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 202, in get Aug 27 02:22:53.056065 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi return self._get("/volumes/%s" % volume_id, "volume") Aug 27 02:22:53.056065 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 297, in _get Aug 27 02:22:53.056065 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.get(url) Aug 27 02:22:53.056065 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 224, in get Aug 27 02:22:53.056065 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'GET', **kwargs) Aug 27 02:22:53.056065 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Aug 27 02:22:53.056065 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Aug 27 02:22:53.056065 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Aug 27 02:22:53.056065 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Aug 27 02:22:53.056065 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.NotFound: Volume 41f7ec3a-cbc1-4f12-8eb3-b34b1f334cca could not be found. (HTTP 404) (Request-ID: req-e01cc9bd-0d4a-4806-aa68-b6d4fbe3563e) Aug 27 02:22:53.056065 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi Aug 27 02:22:53.056065 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 27 02:22:53.056065 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi Aug 27 02:22:53.057292 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 27 02:22:53.057292 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 120, in show Aug 27 02:22:53.057292 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi vol = self.volume_api.get(context, id) Aug 27 02:22:53.057292 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Aug 27 02:22:53.057292 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Aug 27 02:22:53.057292 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 435, in wrapper Aug 27 02:22:53.057292 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi _reraise(exception.VolumeNotFound(volume_id=volume_id)) Aug 27 02:22:53.057292 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise Aug 27 02:22:53.057292 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) Aug 27 02:22:53.057292 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Aug 27 02:22:53.057292 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Aug 27 02:22:53.057292 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 502, in get Aug 27 02:22:53.057292 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi item = cinderclient( Aug 27 02:22:53.057292 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 202, in get Aug 27 02:22:53.057292 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi return self._get("/volumes/%s" % volume_id, "volume") Aug 27 02:22:53.057292 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 297, in _get Aug 27 02:22:53.057292 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.get(url) Aug 27 02:22:53.057292 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 224, in get Aug 27 02:22:53.057292 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'GET', **kwargs) Aug 27 02:22:53.058188 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Aug 27 02:22:53.058188 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Aug 27 02:22:53.058188 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Aug 27 02:22:53.058188 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Aug 27 02:22:53.058188 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi nova.exception.VolumeNotFound: Volume 41f7ec3a-cbc1-4f12-8eb3-b34b1f334cca could not be found. Aug 27 02:22:53.058188 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi Aug 27 02:22:53.058188 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 27 02:22:53.058188 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi Aug 27 02:22:53.058188 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 27 02:22:53.058188 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 27 02:22:53.058188 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 27 02:22:53.058188 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Aug 27 02:22:53.058188 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Aug 27 02:22:53.058188 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Aug 27 02:22:53.058188 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Aug 27 02:22:53.058188 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Aug 27 02:22:53.058188 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Aug 27 02:22:53.058188 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 122, in show Aug 27 02:22:53.058188 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Aug 27 02:22:53.059291 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Volume 41f7ec3a-cbc1-4f12-8eb3-b34b1f334cca could not be found. Aug 27 02:22:53.059291 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi Aug 27 02:22:53.059291 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 27 02:22:53.059291 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi Aug 27 02:22:53.059291 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 27 02:22:53.059291 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 27 02:22:53.059291 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 27 02:22:53.059291 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Aug 27 02:22:53.059291 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Aug 27 02:22:53.059291 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Volume 41f7ec3a-cbc1-4f12-8eb3-b34b1f334cca could not be found. Aug 27 02:22:53.059291 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi Aug 27 02:22:53.059291 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-c2597eb9-fb7e-45b0-ac29-c6451c6ddbe3 tempest-VolumesSnapshotsTestJSON-858074346 tempest-VolumesSnapshotsTestJSON-858074346-project] Returning 404 to user: Volume 41f7ec3a-cbc1-4f12-8eb3-b34b1f334cca could not be found. {{(pid=120863) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Aug 27 02:22:53.059291 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-c2597eb9-fb7e-45b0-ac29-c6451c6ddbe3 tempest-VolumesSnapshotsTestJSON-858074346 tempest-VolumesSnapshotsTestJSON-858074346-project] 10.222.0.52 "GET /compute/v2.1/os-volumes/41f7ec3a-cbc1-4f12-8eb3-b34b1f334cca" status: 404 len: 109 microversion: 2.1 time: 0.404140 Aug 27 02:22:53.059291 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 36/74] 10.222.0.52 () {58 vars in 1245 bytes} [Sat Aug 27 02:22:52 2022] GET /compute/v2.1/os-volumes/41f7ec3a-cbc1-4f12-8eb3-b34b1f334cca => generated 109 bytes in 408 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Aug 27 02:22:53.069833 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-1d0f8a52-0c94-4be2-8ceb-633f0aca1d84 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:22:53.070456 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-1d0f8a52-0c94-4be2-8ceb-633f0aca1d84 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] [instance: 2ad09ab6-ef1f-47a2-bf44-6ae046cc8645] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:22:53.071926 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-e36b4cfa-1c9d-4b97-bc5a-446e77276238 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd945d512ae034ec08838d0a3afde2429', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '880ac4003248441c809a76cfd1b221eb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:22:53.081442 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-1d0f8a52-0c94-4be2-8ceb-633f0aca1d84 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:22:53.082092 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-1d0f8a52-0c94-4be2-8ceb-633f0aca1d84 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:22:53.086179 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-1d0f8a52-0c94-4be2-8ceb-633f0aca1d84 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:22:53.087404 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-1d0f8a52-0c94-4be2-8ceb-633f0aca1d84 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:22:53.090373 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-1d0f8a52-0c94-4be2-8ceb-633f0aca1d84 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:22:53.090979 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-1d0f8a52-0c94-4be2-8ceb-633f0aca1d84 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:22:53.107451 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-e36b4cfa-1c9d-4b97-bc5a-446e77276238 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] 10.222.0.52 "GET /compute/v2.1/servers/05bed8e9-f530-418a-9819-4f65dabdcda0" status: 200 len: 1322 microversion: 2.1 time: 1.059294 Aug 27 02:22:53.122876 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 39/75] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:22:52 2022] GET /compute/v2.1/servers/05bed8e9-f530-418a-9819-4f65dabdcda0 => generated 1322 bytes in 1075 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:22:53.371408 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-25ffb028-51f0-489c-9893-d73ffac1f64f tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Calling method '>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:22:53.372288 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-25ffb028-51f0-489c-9893-d73ffac1f64f tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] [instance: fbef28bb-3cdf-4a27-8493-add7806af828] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:22:53.393602 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-25ffb028-51f0-489c-9893-d73ffac1f64f tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:22:53.396299 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-25ffb028-51f0-489c-9893-d73ffac1f64f tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:22:53.397804 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-25ffb028-51f0-489c-9893-d73ffac1f64f tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:22:53.399795 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-25ffb028-51f0-489c-9893-d73ffac1f64f tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:22:53.400285 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-25ffb028-51f0-489c-9893-d73ffac1f64f tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:22:53.402850 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-25ffb028-51f0-489c-9893-d73ffac1f64f tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:22:53.898779 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-25ffb028-51f0-489c-9893-d73ffac1f64f tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '55640f750d1b49be9c3f79501a63c1ab', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7ef4b44a3f94d599eacce269b06594c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:22:53.916670 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-25ffb028-51f0-489c-9893-d73ffac1f64f tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] 10.222.0.52 "GET /compute/v2.1/servers/fbef28bb-3cdf-4a27-8493-add7806af828" status: 200 len: 1314 microversion: 2.1 time: 0.551736 Aug 27 02:22:53.917601 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 40/76] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:22:53 2022] GET /compute/v2.1/servers/fbef28bb-3cdf-4a27-8493-add7806af828 => generated 1314 bytes in 553 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:22:53.960040 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-1d0f8a52-0c94-4be2-8ceb-633f0aca1d84 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '26a86351583d49d090b3db09f127d49f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5d5ec8450c0941d18d63558baab54737', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:22:53.983088 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-1d0f8a52-0c94-4be2-8ceb-633f0aca1d84 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] 10.222.0.52 "GET /compute/v2.1/servers/2ad09ab6-ef1f-47a2-bf44-6ae046cc8645" status: 200 len: 1366 microversion: 2.1 time: 0.922511 Aug 27 02:22:53.984859 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 37/77] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:22:53 2022] GET /compute/v2.1/servers/2ad09ab6-ef1f-47a2-bf44-6ae046cc8645 => generated 1366 bytes in 925 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:22:54.140661 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-5b725e20-8529-49ed-83e5-81c89d71454d tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Calling method '>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:22:54.141609 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-5b725e20-8529-49ed-83e5-81c89d71454d tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] [instance: 05bed8e9-f530-418a-9819-4f65dabdcda0] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:22:54.148763 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-5b725e20-8529-49ed-83e5-81c89d71454d tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:22:54.149557 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-5b725e20-8529-49ed-83e5-81c89d71454d tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:22:54.150331 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-5b725e20-8529-49ed-83e5-81c89d71454d tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:22:54.152033 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-5b725e20-8529-49ed-83e5-81c89d71454d tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:22:54.152947 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-5b725e20-8529-49ed-83e5-81c89d71454d tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:22:54.153738 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-5b725e20-8529-49ed-83e5-81c89d71454d tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:22:54.436093 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-5b725e20-8529-49ed-83e5-81c89d71454d tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd945d512ae034ec08838d0a3afde2429', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '880ac4003248441c809a76cfd1b221eb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:22:54.446090 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-5b725e20-8529-49ed-83e5-81c89d71454d tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] 10.222.0.52 "GET /compute/v2.1/servers/05bed8e9-f530-418a-9819-4f65dabdcda0" status: 200 len: 1322 microversion: 2.1 time: 0.311354 Aug 27 02:22:54.447251 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 41/78] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:22:54 2022] GET /compute/v2.1/servers/05bed8e9-f530-418a-9819-4f65dabdcda0 => generated 1322 bytes in 314 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:22:54.962971 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-5bc32b4b-0a56-48a1-b87c-a0f5af42f3ad tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:22:54.964259 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-5bc32b4b-0a56-48a1-b87c-a0f5af42f3ad tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] [instance: fbef28bb-3cdf-4a27-8493-add7806af828] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:22:54.976978 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-5bc32b4b-0a56-48a1-b87c-a0f5af42f3ad tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:22:54.977887 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-5bc32b4b-0a56-48a1-b87c-a0f5af42f3ad tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:22:54.978743 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-5bc32b4b-0a56-48a1-b87c-a0f5af42f3ad tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:22:54.980074 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-5bc32b4b-0a56-48a1-b87c-a0f5af42f3ad tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:22:54.981159 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-5bc32b4b-0a56-48a1-b87c-a0f5af42f3ad tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:22:54.981999 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-5bc32b4b-0a56-48a1-b87c-a0f5af42f3ad tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:22:55.014565 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-bcd93af6-67be-4c0e-a98f-47fee72bedb3 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Calling method '>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:22:55.015451 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-bcd93af6-67be-4c0e-a98f-47fee72bedb3 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] [instance: 2ad09ab6-ef1f-47a2-bf44-6ae046cc8645] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:22:55.027592 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-bcd93af6-67be-4c0e-a98f-47fee72bedb3 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:22:55.028406 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-bcd93af6-67be-4c0e-a98f-47fee72bedb3 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:22:55.029188 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-bcd93af6-67be-4c0e-a98f-47fee72bedb3 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:22:55.031042 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-bcd93af6-67be-4c0e-a98f-47fee72bedb3 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:22:55.031747 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-bcd93af6-67be-4c0e-a98f-47fee72bedb3 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:22:55.032227 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-bcd93af6-67be-4c0e-a98f-47fee72bedb3 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:22:55.037152 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG dbcounter [-] [120864] Writing DB stats nova_cell1:SELECT=100 {{(pid=120864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 27 02:22:55.291615 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-5bc32b4b-0a56-48a1-b87c-a0f5af42f3ad tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '55640f750d1b49be9c3f79501a63c1ab', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7ef4b44a3f94d599eacce269b06594c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:22:55.305831 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-5bc32b4b-0a56-48a1-b87c-a0f5af42f3ad tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] 10.222.0.52 "GET /compute/v2.1/servers/fbef28bb-3cdf-4a27-8493-add7806af828" status: 200 len: 1314 microversion: 2.1 time: 0.350302 Aug 27 02:22:55.315665 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 38/79] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:22:54 2022] GET /compute/v2.1/servers/fbef28bb-3cdf-4a27-8493-add7806af828 => generated 1314 bytes in 362 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:22:55.327904 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-bcd93af6-67be-4c0e-a98f-47fee72bedb3 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '26a86351583d49d090b3db09f127d49f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5d5ec8450c0941d18d63558baab54737', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:22:55.342183 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-bcd93af6-67be-4c0e-a98f-47fee72bedb3 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] 10.222.0.52 "GET /compute/v2.1/servers/2ad09ab6-ef1f-47a2-bf44-6ae046cc8645" status: 200 len: 1366 microversion: 2.1 time: 0.332618 Aug 27 02:22:55.343146 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 42/80] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:22:55 2022] GET /compute/v2.1/servers/2ad09ab6-ef1f-47a2-bf44-6ae046cc8645 => generated 1366 bytes in 335 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 27 02:22:55.401448 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [req-d207ec70-b640-45db-982a-9a29276afc0c req-2cd23725-7102-4f8f-b2a6-6c9491e305aa service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "05bed8e9-f530-418a-9819-4f65dabdcda0", "tag": "81921b15-5b92-41f8-a6fa-30be932b9435"}]} {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 27 02:22:55.416860 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [req-d207ec70-b640-45db-982a-9a29276afc0c req-2cd23725-7102-4f8f-b2a6-6c9491e305aa service nova] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:22:55.417341 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [req-d207ec70-b640-45db-982a-9a29276afc0c req-2cd23725-7102-4f8f-b2a6-6c9491e305aa service nova] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:22:55.417770 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [req-d207ec70-b640-45db-982a-9a29276afc0c req-2cd23725-7102-4f8f-b2a6-6c9491e305aa service nova] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:22:55.443072 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.compute.server_external_events [req-d207ec70-b640-45db-982a-9a29276afc0c req-2cd23725-7102-4f8f-b2a6-6c9491e305aa service nova] Creating event network-changed:81921b15-5b92-41f8-a6fa-30be932b9435 for instance 05bed8e9-f530-418a-9819-4f65dabdcda0 on hv-cinder-84630 Aug 27 02:22:55.444013 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [req-d207ec70-b640-45db-982a-9a29276afc0c req-2cd23725-7102-4f8f-b2a6-6c9491e305aa service nova] Acquiring lock "compute-rpcapi-router" {{(pid=120863) lock /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:293}} Aug 27 02:22:55.444846 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [req-d207ec70-b640-45db-982a-9a29276afc0c req-2cd23725-7102-4f8f-b2a6-6c9491e305aa service nova] Acquired lock "compute-rpcapi-router" {{(pid=120863) lock /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:296}} Aug 27 02:22:55.446565 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-7ea3a5c4-7916-4085-8fe1-9cbb0947d51b None None] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:22:55.447345 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-7ea3a5c4-7916-4085-8fe1-9cbb0947d51b None None] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:22:55.448079 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-7ea3a5c4-7916-4085-8fe1-9cbb0947d51b None None] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:22:55.449964 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-7ea3a5c4-7916-4085-8fe1-9cbb0947d51b None None] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:22:55.450778 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-7ea3a5c4-7916-4085-8fe1-9cbb0947d51b None None] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:22:55.451606 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-7ea3a5c4-7916-4085-8fe1-9cbb0947d51b None None] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:22:55.462913 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.compute.rpcapi [None req-7ea3a5c4-7916-4085-8fe1-9cbb0947d51b None None] Automatically selected compute RPC version 6.0 from minimum service version 63 Aug 27 02:22:55.467957 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-7ea3a5c4-7916-4085-8fe1-9cbb0947d51b None None] Releasing lock "compute-rpcapi-router" {{(pid=120863) lock /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:314}} Aug 27 02:22:55.478386 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-83f17bb2-cae6-4d68-af71-dcfe17dfb4c7 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Calling method '>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:22:55.479421 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-83f17bb2-cae6-4d68-af71-dcfe17dfb4c7 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] [instance: 05bed8e9-f530-418a-9819-4f65dabdcda0] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:22:55.494294 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-7ea3a5c4-7916-4085-8fe1-9cbb0947d51b None None] 10.222.0.52 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.176696 Aug 27 02:22:55.495684 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 39/81] 10.222.0.52 () {64 vars in 1317 bytes} [Sat Aug 27 02:22:55 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 179 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 27 02:22:55.498205 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-83f17bb2-cae6-4d68-af71-dcfe17dfb4c7 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:22:55.499435 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-83f17bb2-cae6-4d68-af71-dcfe17dfb4c7 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:22:55.500388 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-83f17bb2-cae6-4d68-af71-dcfe17dfb4c7 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:22:55.501443 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-83f17bb2-cae6-4d68-af71-dcfe17dfb4c7 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:22:55.501907 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-83f17bb2-cae6-4d68-af71-dcfe17dfb4c7 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:22:55.502695 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-83f17bb2-cae6-4d68-af71-dcfe17dfb4c7 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:22:55.691285 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-83f17bb2-cae6-4d68-af71-dcfe17dfb4c7 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd945d512ae034ec08838d0a3afde2429', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '880ac4003248441c809a76cfd1b221eb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:22:55.700503 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-83f17bb2-cae6-4d68-af71-dcfe17dfb4c7 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] 10.222.0.52 "GET /compute/v2.1/servers/05bed8e9-f530-418a-9819-4f65dabdcda0" status: 200 len: 1322 microversion: 2.1 time: 0.231120 Aug 27 02:22:55.701068 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 43/82] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:22:55 2022] GET /compute/v2.1/servers/05bed8e9-f530-418a-9819-4f65dabdcda0 => generated 1322 bytes in 233 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:22:56.346466 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-5dcb1e85-6433-4f5a-89fc-f2cc7dca1ecd tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:22:56.347169 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-5dcb1e85-6433-4f5a-89fc-f2cc7dca1ecd tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] [instance: fbef28bb-3cdf-4a27-8493-add7806af828] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:22:56.354534 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-5dcb1e85-6433-4f5a-89fc-f2cc7dca1ecd tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:22:56.355028 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-5dcb1e85-6433-4f5a-89fc-f2cc7dca1ecd tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:22:56.355454 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-5dcb1e85-6433-4f5a-89fc-f2cc7dca1ecd tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:22:56.356146 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-5dcb1e85-6433-4f5a-89fc-f2cc7dca1ecd tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:22:56.356589 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-5dcb1e85-6433-4f5a-89fc-f2cc7dca1ecd tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:22:56.357016 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-5dcb1e85-6433-4f5a-89fc-f2cc7dca1ecd tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:22:56.373938 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-3c515e69-ff6e-496d-8490-7760665a6907 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Calling method '>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:22:56.374713 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-3c515e69-ff6e-496d-8490-7760665a6907 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] [instance: 2ad09ab6-ef1f-47a2-bf44-6ae046cc8645] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:22:56.384607 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-3c515e69-ff6e-496d-8490-7760665a6907 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:22:56.385425 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-3c515e69-ff6e-496d-8490-7760665a6907 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:22:56.386207 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-3c515e69-ff6e-496d-8490-7760665a6907 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:22:56.387333 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-3c515e69-ff6e-496d-8490-7760665a6907 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:22:56.387874 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-3c515e69-ff6e-496d-8490-7760665a6907 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:22:56.388307 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-3c515e69-ff6e-496d-8490-7760665a6907 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:22:56.571321 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-5dcb1e85-6433-4f5a-89fc-f2cc7dca1ecd tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '55640f750d1b49be9c3f79501a63c1ab', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7ef4b44a3f94d599eacce269b06594c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:22:56.593543 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-5dcb1e85-6433-4f5a-89fc-f2cc7dca1ecd tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] 10.222.0.52 "GET /compute/v2.1/servers/fbef28bb-3cdf-4a27-8493-add7806af828" status: 200 len: 1314 microversion: 2.1 time: 0.253662 Aug 27 02:22:56.609151 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 40/83] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:22:56 2022] GET /compute/v2.1/servers/fbef28bb-3cdf-4a27-8493-add7806af828 => generated 1314 bytes in 257 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:22:56.609869 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-3c515e69-ff6e-496d-8490-7760665a6907 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '26a86351583d49d090b3db09f127d49f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5d5ec8450c0941d18d63558baab54737', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:22:56.617254 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-3c515e69-ff6e-496d-8490-7760665a6907 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] 10.222.0.52 "GET /compute/v2.1/servers/2ad09ab6-ef1f-47a2-bf44-6ae046cc8645" status: 200 len: 1366 microversion: 2.1 time: 0.247173 Aug 27 02:22:56.618474 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 44/84] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:22:56 2022] GET /compute/v2.1/servers/2ad09ab6-ef1f-47a2-bf44-6ae046cc8645 => generated 1366 bytes in 249 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:22:56.728081 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-83671b44-46b9-4e46-8658-a8af336893e1 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:22:56.728898 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-83671b44-46b9-4e46-8658-a8af336893e1 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] [instance: 05bed8e9-f530-418a-9819-4f65dabdcda0] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:22:56.740753 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-83671b44-46b9-4e46-8658-a8af336893e1 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:22:56.741310 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-83671b44-46b9-4e46-8658-a8af336893e1 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:22:56.741814 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-83671b44-46b9-4e46-8658-a8af336893e1 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:22:56.742508 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-83671b44-46b9-4e46-8658-a8af336893e1 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:22:56.743383 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-83671b44-46b9-4e46-8658-a8af336893e1 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:22:56.743383 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-83671b44-46b9-4e46-8658-a8af336893e1 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:22:56.910536 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-83671b44-46b9-4e46-8658-a8af336893e1 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd945d512ae034ec08838d0a3afde2429', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '880ac4003248441c809a76cfd1b221eb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:22:56.918900 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-83671b44-46b9-4e46-8658-a8af336893e1 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] 10.222.0.52 "GET /compute/v2.1/servers/05bed8e9-f530-418a-9819-4f65dabdcda0" status: 200 len: 1322 microversion: 2.1 time: 0.200669 Aug 27 02:22:56.919483 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 41/85] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:22:56 2022] GET /compute/v2.1/servers/05bed8e9-f530-418a-9819-4f65dabdcda0 => generated 1322 bytes in 202 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:22:57.626693 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-13536146-26d5-4ee3-8a0c-3038de9f3189 tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Calling method '>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:22:57.628373 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-13536146-26d5-4ee3-8a0c-3038de9f3189 tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] [instance: fbef28bb-3cdf-4a27-8493-add7806af828] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:22:57.634428 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG dbcounter [-] [120864] Writing DB stats nova_api:SELECT=71,nova_api:INSERT=4 {{(pid=120864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 27 02:22:57.640419 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-64ba8a60-b5b8-4528-939e-17a92cfa079c tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:22:57.641240 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-64ba8a60-b5b8-4528-939e-17a92cfa079c tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] [instance: 2ad09ab6-ef1f-47a2-bf44-6ae046cc8645] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:22:57.647987 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-13536146-26d5-4ee3-8a0c-3038de9f3189 tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:22:57.650699 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-13536146-26d5-4ee3-8a0c-3038de9f3189 tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:22:57.652560 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-64ba8a60-b5b8-4528-939e-17a92cfa079c tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:22:57.653277 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-64ba8a60-b5b8-4528-939e-17a92cfa079c tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:22:57.653902 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-13536146-26d5-4ee3-8a0c-3038de9f3189 tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:22:57.654472 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-64ba8a60-b5b8-4528-939e-17a92cfa079c tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:22:57.657331 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-13536146-26d5-4ee3-8a0c-3038de9f3189 tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:22:57.657331 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-13536146-26d5-4ee3-8a0c-3038de9f3189 tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:22:57.657331 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-13536146-26d5-4ee3-8a0c-3038de9f3189 tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:22:57.658545 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-64ba8a60-b5b8-4528-939e-17a92cfa079c tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:22:57.659134 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-64ba8a60-b5b8-4528-939e-17a92cfa079c tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:22:57.659616 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-64ba8a60-b5b8-4528-939e-17a92cfa079c tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:22:57.939472 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-64ba8a60-b5b8-4528-939e-17a92cfa079c tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '26a86351583d49d090b3db09f127d49f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5d5ec8450c0941d18d63558baab54737', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:22:57.963130 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-64ba8a60-b5b8-4528-939e-17a92cfa079c tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] 10.222.0.52 "GET /compute/v2.1/servers/2ad09ab6-ef1f-47a2-bf44-6ae046cc8645" status: 200 len: 1366 microversion: 2.1 time: 0.326657 Aug 27 02:22:57.964136 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 42/86] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:22:57 2022] GET /compute/v2.1/servers/2ad09ab6-ef1f-47a2-bf44-6ae046cc8645 => generated 1366 bytes in 329 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 27 02:22:57.969739 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-43849649-8af0-40f5-893b-73c0d1d124dd tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:22:57.970326 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-43849649-8af0-40f5-893b-73c0d1d124dd tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] [instance: 05bed8e9-f530-418a-9819-4f65dabdcda0] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:22:57.982909 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-43849649-8af0-40f5-893b-73c0d1d124dd tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:22:57.993194 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-43849649-8af0-40f5-893b-73c0d1d124dd tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:22:57.995375 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-43849649-8af0-40f5-893b-73c0d1d124dd tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.011s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:22:57.996820 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-43849649-8af0-40f5-893b-73c0d1d124dd tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:22:58.004169 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-43849649-8af0-40f5-893b-73c0d1d124dd tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.007s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:22:58.006263 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-43849649-8af0-40f5-893b-73c0d1d124dd tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:22:58.017231 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-13536146-26d5-4ee3-8a0c-3038de9f3189 tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '55640f750d1b49be9c3f79501a63c1ab', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7ef4b44a3f94d599eacce269b06594c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:22:58.040473 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-13536146-26d5-4ee3-8a0c-3038de9f3189 tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] 10.222.0.52 "GET /compute/v2.1/servers/fbef28bb-3cdf-4a27-8493-add7806af828" status: 200 len: 1314 microversion: 2.1 time: 0.418135 Aug 27 02:22:58.045239 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 45/87] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:22:57 2022] GET /compute/v2.1/servers/fbef28bb-3cdf-4a27-8493-add7806af828 => generated 1314 bytes in 424 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 27 02:22:58.215668 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-43849649-8af0-40f5-893b-73c0d1d124dd tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd945d512ae034ec08838d0a3afde2429', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '880ac4003248441c809a76cfd1b221eb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:22:58.226482 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-43849649-8af0-40f5-893b-73c0d1d124dd tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] 10.222.0.52 "GET /compute/v2.1/servers/05bed8e9-f530-418a-9819-4f65dabdcda0" status: 200 len: 1322 microversion: 2.1 time: 0.260422 Aug 27 02:22:58.227055 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 43/88] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:22:57 2022] GET /compute/v2.1/servers/05bed8e9-f530-418a-9819-4f65dabdcda0 => generated 1322 bytes in 262 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:22:58.999719 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-92e96577-d420-481c-a26b-7a7bf718cead tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Calling method '>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:22:59.000838 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-92e96577-d420-481c-a26b-7a7bf718cead tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] [instance: 2ad09ab6-ef1f-47a2-bf44-6ae046cc8645] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:22:59.013725 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-92e96577-d420-481c-a26b-7a7bf718cead tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:22:59.014829 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-92e96577-d420-481c-a26b-7a7bf718cead tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:22:59.016333 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-92e96577-d420-481c-a26b-7a7bf718cead tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:22:59.018252 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-92e96577-d420-481c-a26b-7a7bf718cead tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:22:59.018940 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-92e96577-d420-481c-a26b-7a7bf718cead tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:22:59.020105 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-92e96577-d420-481c-a26b-7a7bf718cead tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:22:59.074241 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-3a5f46d5-d5be-401f-91f9-405f0f162923 tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:22:59.075414 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-3a5f46d5-d5be-401f-91f9-405f0f162923 tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] [instance: fbef28bb-3cdf-4a27-8493-add7806af828] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:22:59.111668 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-3a5f46d5-d5be-401f-91f9-405f0f162923 tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:22:59.113980 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-3a5f46d5-d5be-401f-91f9-405f0f162923 tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:22:59.114427 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-3a5f46d5-d5be-401f-91f9-405f0f162923 tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:22:59.115376 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-3a5f46d5-d5be-401f-91f9-405f0f162923 tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:22:59.115902 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-3a5f46d5-d5be-401f-91f9-405f0f162923 tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:22:59.116331 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-3a5f46d5-d5be-401f-91f9-405f0f162923 tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:22:59.270073 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-92e96577-d420-481c-a26b-7a7bf718cead tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '26a86351583d49d090b3db09f127d49f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5d5ec8450c0941d18d63558baab54737', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:22:59.278013 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-92e96577-d420-481c-a26b-7a7bf718cead tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] 10.222.0.52 "GET /compute/v2.1/servers/2ad09ab6-ef1f-47a2-bf44-6ae046cc8645" status: 200 len: 1366 microversion: 2.1 time: 0.286135 Aug 27 02:22:59.278571 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 46/89] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:22:58 2022] GET /compute/v2.1/servers/2ad09ab6-ef1f-47a2-bf44-6ae046cc8645 => generated 1366 bytes in 289 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:22:59.283108 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-40115e6e-8462-4656-b1b9-9a5486e92ab6 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Calling method '>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:22:59.283695 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-40115e6e-8462-4656-b1b9-9a5486e92ab6 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] [instance: 05bed8e9-f530-418a-9819-4f65dabdcda0] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:22:59.289904 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-40115e6e-8462-4656-b1b9-9a5486e92ab6 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:22:59.290456 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-40115e6e-8462-4656-b1b9-9a5486e92ab6 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:22:59.290951 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-40115e6e-8462-4656-b1b9-9a5486e92ab6 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:22:59.291697 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-40115e6e-8462-4656-b1b9-9a5486e92ab6 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:22:59.292136 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-40115e6e-8462-4656-b1b9-9a5486e92ab6 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:22:59.292547 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-40115e6e-8462-4656-b1b9-9a5486e92ab6 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:22:59.320134 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-3a5f46d5-d5be-401f-91f9-405f0f162923 tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '55640f750d1b49be9c3f79501a63c1ab', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7ef4b44a3f94d599eacce269b06594c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:22:59.330242 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-3a5f46d5-d5be-401f-91f9-405f0f162923 tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] 10.222.0.52 "GET /compute/v2.1/servers/fbef28bb-3cdf-4a27-8493-add7806af828" status: 200 len: 1314 microversion: 2.1 time: 0.262477 Aug 27 02:22:59.331132 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 44/90] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:22:59 2022] GET /compute/v2.1/servers/fbef28bb-3cdf-4a27-8493-add7806af828 => generated 1314 bytes in 264 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:22:59.484247 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-40115e6e-8462-4656-b1b9-9a5486e92ab6 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd945d512ae034ec08838d0a3afde2429', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '880ac4003248441c809a76cfd1b221eb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:22:59.501757 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-40115e6e-8462-4656-b1b9-9a5486e92ab6 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] 10.222.0.52 "GET /compute/v2.1/servers/05bed8e9-f530-418a-9819-4f65dabdcda0" status: 200 len: 1322 microversion: 2.1 time: 0.222111 Aug 27 02:22:59.502457 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 47/91] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:22:59 2022] GET /compute/v2.1/servers/05bed8e9-f530-418a-9819-4f65dabdcda0 => generated 1322 bytes in 223 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:23:00.299877 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-32935a94-42c5-4ea3-b602-d8c0d435290f tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:23:00.300575 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-32935a94-42c5-4ea3-b602-d8c0d435290f tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] [instance: 2ad09ab6-ef1f-47a2-bf44-6ae046cc8645] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:23:00.307496 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-32935a94-42c5-4ea3-b602-d8c0d435290f tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:23:00.308123 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-32935a94-42c5-4ea3-b602-d8c0d435290f tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:23:00.308547 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-32935a94-42c5-4ea3-b602-d8c0d435290f tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:23:00.309351 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-32935a94-42c5-4ea3-b602-d8c0d435290f tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:23:00.309792 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-32935a94-42c5-4ea3-b602-d8c0d435290f tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:23:00.310182 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-32935a94-42c5-4ea3-b602-d8c0d435290f tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:23:00.352056 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-9f6add40-d225-4cc4-bf06-f9cb60d4a3d2 tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Calling method '>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:23:00.352690 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-9f6add40-d225-4cc4-bf06-f9cb60d4a3d2 tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] [instance: fbef28bb-3cdf-4a27-8493-add7806af828] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:23:00.359046 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-9f6add40-d225-4cc4-bf06-f9cb60d4a3d2 tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:23:00.359556 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-9f6add40-d225-4cc4-bf06-f9cb60d4a3d2 tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:23:00.360016 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-9f6add40-d225-4cc4-bf06-f9cb60d4a3d2 tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:23:00.360791 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-9f6add40-d225-4cc4-bf06-f9cb60d4a3d2 tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:23:00.361288 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-9f6add40-d225-4cc4-bf06-f9cb60d4a3d2 tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:23:00.361712 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-9f6add40-d225-4cc4-bf06-f9cb60d4a3d2 tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:23:00.500648 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-32935a94-42c5-4ea3-b602-d8c0d435290f tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '26a86351583d49d090b3db09f127d49f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5d5ec8450c0941d18d63558baab54737', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:23:00.511657 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-32935a94-42c5-4ea3-b602-d8c0d435290f tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] 10.222.0.52 "GET /compute/v2.1/servers/2ad09ab6-ef1f-47a2-bf44-6ae046cc8645" status: 200 len: 1366 microversion: 2.1 time: 0.215794 Aug 27 02:23:00.512429 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 45/92] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:23:00 2022] GET /compute/v2.1/servers/2ad09ab6-ef1f-47a2-bf44-6ae046cc8645 => generated 1366 bytes in 218 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:23:00.524539 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-fb1d5ecc-2b33-4757-b4c4-360ea64e117a tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:23:00.525409 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-fb1d5ecc-2b33-4757-b4c4-360ea64e117a tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] [instance: 05bed8e9-f530-418a-9819-4f65dabdcda0] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:23:00.531728 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-fb1d5ecc-2b33-4757-b4c4-360ea64e117a tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:23:00.532216 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-fb1d5ecc-2b33-4757-b4c4-360ea64e117a tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:23:00.532651 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-fb1d5ecc-2b33-4757-b4c4-360ea64e117a tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:23:00.533391 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-fb1d5ecc-2b33-4757-b4c4-360ea64e117a tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:23:00.533848 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-fb1d5ecc-2b33-4757-b4c4-360ea64e117a tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:23:00.534268 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-fb1d5ecc-2b33-4757-b4c4-360ea64e117a tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:23:00.551052 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-9f6add40-d225-4cc4-bf06-f9cb60d4a3d2 tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '55640f750d1b49be9c3f79501a63c1ab', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7ef4b44a3f94d599eacce269b06594c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:23:00.560178 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-9f6add40-d225-4cc4-bf06-f9cb60d4a3d2 tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] 10.222.0.52 "GET /compute/v2.1/servers/fbef28bb-3cdf-4a27-8493-add7806af828" status: 200 len: 1314 microversion: 2.1 time: 0.212092 Aug 27 02:23:00.560919 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 48/93] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:23:00 2022] GET /compute/v2.1/servers/fbef28bb-3cdf-4a27-8493-add7806af828 => generated 1314 bytes in 214 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 27 02:23:00.681811 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-fb1d5ecc-2b33-4757-b4c4-360ea64e117a tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd945d512ae034ec08838d0a3afde2429', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '880ac4003248441c809a76cfd1b221eb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:23:00.689737 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-fb1d5ecc-2b33-4757-b4c4-360ea64e117a tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] 10.222.0.52 "GET /compute/v2.1/servers/05bed8e9-f530-418a-9819-4f65dabdcda0" status: 200 len: 1322 microversion: 2.1 time: 0.169039 Aug 27 02:23:00.690346 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 46/94] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:23:00 2022] GET /compute/v2.1/servers/05bed8e9-f530-418a-9819-4f65dabdcda0 => generated 1322 bytes in 170 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 27 02:23:01.533078 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-04ae231d-3c2c-4969-99a7-602f111357bd tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Calling method '>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:23:01.533708 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-04ae231d-3c2c-4969-99a7-602f111357bd tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] [instance: 2ad09ab6-ef1f-47a2-bf44-6ae046cc8645] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:23:01.539741 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-04ae231d-3c2c-4969-99a7-602f111357bd tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:23:01.540224 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-04ae231d-3c2c-4969-99a7-602f111357bd tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:23:01.540655 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-04ae231d-3c2c-4969-99a7-602f111357bd tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:23:01.541382 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-04ae231d-3c2c-4969-99a7-602f111357bd tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:23:01.541831 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-04ae231d-3c2c-4969-99a7-602f111357bd tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:23:01.542245 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-04ae231d-3c2c-4969-99a7-602f111357bd tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:23:01.587965 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-86af5509-1b17-46c5-9978-e39b5821faf1 tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:23:01.587965 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-86af5509-1b17-46c5-9978-e39b5821faf1 tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] [instance: fbef28bb-3cdf-4a27-8493-add7806af828] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:23:01.593037 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-86af5509-1b17-46c5-9978-e39b5821faf1 tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:23:01.593568 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-86af5509-1b17-46c5-9978-e39b5821faf1 tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:23:01.594001 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-86af5509-1b17-46c5-9978-e39b5821faf1 tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:23:01.594714 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-86af5509-1b17-46c5-9978-e39b5821faf1 tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:23:01.595174 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-86af5509-1b17-46c5-9978-e39b5821faf1 tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:23:01.595841 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-86af5509-1b17-46c5-9978-e39b5821faf1 tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:23:01.787785 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-04ae231d-3c2c-4969-99a7-602f111357bd tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '26a86351583d49d090b3db09f127d49f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5d5ec8450c0941d18d63558baab54737', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:23:01.801030 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-04ae231d-3c2c-4969-99a7-602f111357bd tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] 10.222.0.52 "GET /compute/v2.1/servers/2ad09ab6-ef1f-47a2-bf44-6ae046cc8645" status: 200 len: 1366 microversion: 2.1 time: 0.271329 Aug 27 02:23:01.809215 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 49/95] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:23:01 2022] GET /compute/v2.1/servers/2ad09ab6-ef1f-47a2-bf44-6ae046cc8645 => generated 1366 bytes in 274 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 27 02:23:01.810246 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-0c56b2cb-71d2-4550-8dd5-7f322f530aac tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Calling method '>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:23:01.811394 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-0c56b2cb-71d2-4550-8dd5-7f322f530aac tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] [instance: 05bed8e9-f530-418a-9819-4f65dabdcda0] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:23:01.819925 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-86af5509-1b17-46c5-9978-e39b5821faf1 tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '55640f750d1b49be9c3f79501a63c1ab', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7ef4b44a3f94d599eacce269b06594c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:23:01.820190 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-0c56b2cb-71d2-4550-8dd5-7f322f530aac tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:23:01.820638 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-0c56b2cb-71d2-4550-8dd5-7f322f530aac tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:23:01.821064 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-0c56b2cb-71d2-4550-8dd5-7f322f530aac tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:23:01.821813 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-0c56b2cb-71d2-4550-8dd5-7f322f530aac tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:23:01.822258 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-0c56b2cb-71d2-4550-8dd5-7f322f530aac tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:23:01.822669 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-0c56b2cb-71d2-4550-8dd5-7f322f530aac tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:23:01.831821 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-86af5509-1b17-46c5-9978-e39b5821faf1 tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] 10.222.0.52 "GET /compute/v2.1/servers/fbef28bb-3cdf-4a27-8493-add7806af828" status: 200 len: 1314 microversion: 2.1 time: 0.252239 Aug 27 02:23:01.832425 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 47/96] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:23:01 2022] GET /compute/v2.1/servers/fbef28bb-3cdf-4a27-8493-add7806af828 => generated 1314 bytes in 254 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 27 02:23:01.969852 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-0c56b2cb-71d2-4550-8dd5-7f322f530aac tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd945d512ae034ec08838d0a3afde2429', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '880ac4003248441c809a76cfd1b221eb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:23:01.977857 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-0c56b2cb-71d2-4550-8dd5-7f322f530aac tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] 10.222.0.52 "GET /compute/v2.1/servers/05bed8e9-f530-418a-9819-4f65dabdcda0" status: 200 len: 1322 microversion: 2.1 time: 0.174145 Aug 27 02:23:01.978443 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 50/97] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:23:01 2022] GET /compute/v2.1/servers/05bed8e9-f530-418a-9819-4f65dabdcda0 => generated 1322 bytes in 176 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:23:02.824670 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-cdaedbaa-97b8-4c51-960f-a80214a58041 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:23:02.825498 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-cdaedbaa-97b8-4c51-960f-a80214a58041 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] [instance: 2ad09ab6-ef1f-47a2-bf44-6ae046cc8645] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:23:02.832694 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-cdaedbaa-97b8-4c51-960f-a80214a58041 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:23:02.832694 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-cdaedbaa-97b8-4c51-960f-a80214a58041 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:23:02.832694 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-cdaedbaa-97b8-4c51-960f-a80214a58041 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:23:02.837565 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-cdaedbaa-97b8-4c51-960f-a80214a58041 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:23:02.838234 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-cdaedbaa-97b8-4c51-960f-a80214a58041 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:23:02.838841 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-cdaedbaa-97b8-4c51-960f-a80214a58041 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:23:02.863490 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-90416e41-ff4c-4efa-bb75-f9302a7cbb7c tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Calling method '>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:23:02.864352 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-90416e41-ff4c-4efa-bb75-f9302a7cbb7c tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] [instance: fbef28bb-3cdf-4a27-8493-add7806af828] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:23:02.871037 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-90416e41-ff4c-4efa-bb75-f9302a7cbb7c tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:23:02.871496 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-90416e41-ff4c-4efa-bb75-f9302a7cbb7c tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:23:02.871901 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-90416e41-ff4c-4efa-bb75-f9302a7cbb7c tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:23:02.872592 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-90416e41-ff4c-4efa-bb75-f9302a7cbb7c tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:23:02.873037 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-90416e41-ff4c-4efa-bb75-f9302a7cbb7c tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:23:02.873491 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-90416e41-ff4c-4efa-bb75-f9302a7cbb7c tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:23:03.047803 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-90416e41-ff4c-4efa-bb75-f9302a7cbb7c tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '55640f750d1b49be9c3f79501a63c1ab', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7ef4b44a3f94d599eacce269b06594c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:23:03.050192 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-cdaedbaa-97b8-4c51-960f-a80214a58041 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '26a86351583d49d090b3db09f127d49f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5d5ec8450c0941d18d63558baab54737', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:23:03.058279 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-cdaedbaa-97b8-4c51-960f-a80214a58041 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] 10.222.0.52 "GET /compute/v2.1/servers/2ad09ab6-ef1f-47a2-bf44-6ae046cc8645" status: 200 len: 1366 microversion: 2.1 time: 0.237739 Aug 27 02:23:03.058472 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-90416e41-ff4c-4efa-bb75-f9302a7cbb7c tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] 10.222.0.52 "GET /compute/v2.1/servers/fbef28bb-3cdf-4a27-8493-add7806af828" status: 200 len: 1314 microversion: 2.1 time: 0.198421 Aug 27 02:23:03.058972 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 48/98] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:23:02 2022] GET /compute/v2.1/servers/2ad09ab6-ef1f-47a2-bf44-6ae046cc8645 => generated 1366 bytes in 239 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:23:03.064593 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 51/99] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:23:02 2022] GET /compute/v2.1/servers/fbef28bb-3cdf-4a27-8493-add7806af828 => generated 1314 bytes in 200 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:23:03.064793 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-044fd5bd-3ac2-485d-8a74-a62d7a3b3526 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:23:03.064793 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-044fd5bd-3ac2-485d-8a74-a62d7a3b3526 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] [instance: 05bed8e9-f530-418a-9819-4f65dabdcda0] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:23:03.071013 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-044fd5bd-3ac2-485d-8a74-a62d7a3b3526 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:23:03.071499 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-044fd5bd-3ac2-485d-8a74-a62d7a3b3526 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:23:03.071977 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-044fd5bd-3ac2-485d-8a74-a62d7a3b3526 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:23:03.072673 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-044fd5bd-3ac2-485d-8a74-a62d7a3b3526 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:23:03.073146 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-044fd5bd-3ac2-485d-8a74-a62d7a3b3526 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:23:03.073937 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-044fd5bd-3ac2-485d-8a74-a62d7a3b3526 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:23:03.215495 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-044fd5bd-3ac2-485d-8a74-a62d7a3b3526 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd945d512ae034ec08838d0a3afde2429', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '880ac4003248441c809a76cfd1b221eb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:23:03.223541 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-044fd5bd-3ac2-485d-8a74-a62d7a3b3526 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] 10.222.0.52 "GET /compute/v2.1/servers/05bed8e9-f530-418a-9819-4f65dabdcda0" status: 200 len: 1322 microversion: 2.1 time: 0.163717 Aug 27 02:23:03.224282 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 49/100] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:23:03 2022] GET /compute/v2.1/servers/05bed8e9-f530-418a-9819-4f65dabdcda0 => generated 1322 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:23:04.082784 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-2e87aa2e-134f-4d0d-bb23-efc9040611d6 tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:23:04.088217 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-5ebe6608-8588-44f6-a5d3-196e4cb5bf47 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Calling method '>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:23:04.090093 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-2e87aa2e-134f-4d0d-bb23-efc9040611d6 tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] [instance: fbef28bb-3cdf-4a27-8493-add7806af828] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:23:04.090951 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-5ebe6608-8588-44f6-a5d3-196e4cb5bf47 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] [instance: 2ad09ab6-ef1f-47a2-bf44-6ae046cc8645] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:23:04.101189 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-2e87aa2e-134f-4d0d-bb23-efc9040611d6 tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:23:04.101952 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-2e87aa2e-134f-4d0d-bb23-efc9040611d6 tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:23:04.102582 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-2e87aa2e-134f-4d0d-bb23-efc9040611d6 tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:23:04.103533 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-2e87aa2e-134f-4d0d-bb23-efc9040611d6 tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:23:04.104055 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-2e87aa2e-134f-4d0d-bb23-efc9040611d6 tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:23:04.104693 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-2e87aa2e-134f-4d0d-bb23-efc9040611d6 tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:23:04.107563 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-5ebe6608-8588-44f6-a5d3-196e4cb5bf47 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:23:04.108176 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-5ebe6608-8588-44f6-a5d3-196e4cb5bf47 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:23:04.108884 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-5ebe6608-8588-44f6-a5d3-196e4cb5bf47 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:23:04.110006 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-5ebe6608-8588-44f6-a5d3-196e4cb5bf47 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:23:04.110723 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-5ebe6608-8588-44f6-a5d3-196e4cb5bf47 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:23:04.111441 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-5ebe6608-8588-44f6-a5d3-196e4cb5bf47 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:23:04.132773 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG dbcounter [-] [120863] Writing DB stats nova_cell1:SELECT=100 {{(pid=120863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 27 02:23:04.331027 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-2e87aa2e-134f-4d0d-bb23-efc9040611d6 tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '55640f750d1b49be9c3f79501a63c1ab', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7ef4b44a3f94d599eacce269b06594c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:23:04.344118 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-2e87aa2e-134f-4d0d-bb23-efc9040611d6 tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] 10.222.0.52 "GET /compute/v2.1/servers/fbef28bb-3cdf-4a27-8493-add7806af828" status: 200 len: 1314 microversion: 2.1 time: 0.265317 Aug 27 02:23:04.345512 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 50/101] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:23:04 2022] GET /compute/v2.1/servers/fbef28bb-3cdf-4a27-8493-add7806af828 => generated 1314 bytes in 268 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:23:04.350553 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-dd3642b7-938d-4f50-bacb-9b34321dc5aa tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:23:04.351157 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-dd3642b7-938d-4f50-bacb-9b34321dc5aa tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] [instance: 05bed8e9-f530-418a-9819-4f65dabdcda0] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:23:04.352801 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-5ebe6608-8588-44f6-a5d3-196e4cb5bf47 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '26a86351583d49d090b3db09f127d49f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5d5ec8450c0941d18d63558baab54737', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:23:04.357923 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-dd3642b7-938d-4f50-bacb-9b34321dc5aa tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:23:04.358524 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-dd3642b7-938d-4f50-bacb-9b34321dc5aa tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:23:04.359052 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-dd3642b7-938d-4f50-bacb-9b34321dc5aa tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:23:04.359805 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-dd3642b7-938d-4f50-bacb-9b34321dc5aa tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:23:04.360477 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-dd3642b7-938d-4f50-bacb-9b34321dc5aa tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:23:04.360932 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-dd3642b7-938d-4f50-bacb-9b34321dc5aa tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:23:04.365048 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-5ebe6608-8588-44f6-a5d3-196e4cb5bf47 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] 10.222.0.52 "GET /compute/v2.1/servers/2ad09ab6-ef1f-47a2-bf44-6ae046cc8645" status: 200 len: 1366 microversion: 2.1 time: 0.287153 Aug 27 02:23:04.366810 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 52/102] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:23:04 2022] GET /compute/v2.1/servers/2ad09ab6-ef1f-47a2-bf44-6ae046cc8645 => generated 1366 bytes in 290 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:23:04.520538 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-dd3642b7-938d-4f50-bacb-9b34321dc5aa tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd945d512ae034ec08838d0a3afde2429', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '880ac4003248441c809a76cfd1b221eb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:23:04.537416 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-dd3642b7-938d-4f50-bacb-9b34321dc5aa tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] 10.222.0.52 "GET /compute/v2.1/servers/05bed8e9-f530-418a-9819-4f65dabdcda0" status: 200 len: 1322 microversion: 2.1 time: 0.181770 Aug 27 02:23:04.538203 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 51/103] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:23:04 2022] GET /compute/v2.1/servers/05bed8e9-f530-418a-9819-4f65dabdcda0 => generated 1322 bytes in 192 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:23:05.372525 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-9a7b5e12-88f9-47ea-a429-845ec43e8f8c tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Calling method '>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:23:05.373353 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-9a7b5e12-88f9-47ea-a429-845ec43e8f8c tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] [instance: fbef28bb-3cdf-4a27-8493-add7806af828] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:23:05.379929 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-9a7b5e12-88f9-47ea-a429-845ec43e8f8c tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:23:05.380493 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-9a7b5e12-88f9-47ea-a429-845ec43e8f8c tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:23:05.381194 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-9a7b5e12-88f9-47ea-a429-845ec43e8f8c tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:23:05.382007 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-9a7b5e12-88f9-47ea-a429-845ec43e8f8c tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:23:05.382455 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-9a7b5e12-88f9-47ea-a429-845ec43e8f8c tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:23:05.382867 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-9a7b5e12-88f9-47ea-a429-845ec43e8f8c tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:23:05.391608 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-c8cdcbfc-98bd-448d-ac6d-01f4ca603cbd tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:23:05.393454 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-c8cdcbfc-98bd-448d-ac6d-01f4ca603cbd tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] [instance: 2ad09ab6-ef1f-47a2-bf44-6ae046cc8645] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:23:05.399178 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-c8cdcbfc-98bd-448d-ac6d-01f4ca603cbd tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:23:05.399666 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-c8cdcbfc-98bd-448d-ac6d-01f4ca603cbd tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:23:05.400103 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-c8cdcbfc-98bd-448d-ac6d-01f4ca603cbd tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:23:05.400840 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-c8cdcbfc-98bd-448d-ac6d-01f4ca603cbd tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:23:05.401345 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-c8cdcbfc-98bd-448d-ac6d-01f4ca603cbd tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:23:05.401767 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-c8cdcbfc-98bd-448d-ac6d-01f4ca603cbd tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:23:05.460843 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG dbcounter [-] [120863] Writing DB stats nova_cell0:SELECT=2 {{(pid=120863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 27 02:23:05.736893 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-9a7b5e12-88f9-47ea-a429-845ec43e8f8c tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '55640f750d1b49be9c3f79501a63c1ab', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7ef4b44a3f94d599eacce269b06594c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:23:05.752393 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-9a7b5e12-88f9-47ea-a429-845ec43e8f8c tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] 10.222.0.52 "GET /compute/v2.1/servers/fbef28bb-3cdf-4a27-8493-add7806af828" status: 200 len: 1314 microversion: 2.1 time: 0.385795 Aug 27 02:23:05.766471 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 53/104] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:23:05 2022] GET /compute/v2.1/servers/fbef28bb-3cdf-4a27-8493-add7806af828 => generated 1314 bytes in 387 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:23:05.766688 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-c8cdcbfc-98bd-448d-ac6d-01f4ca603cbd tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '26a86351583d49d090b3db09f127d49f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5d5ec8450c0941d18d63558baab54737', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:23:05.770271 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-c8cdcbfc-98bd-448d-ac6d-01f4ca603cbd tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] 10.222.0.52 "GET /compute/v2.1/servers/2ad09ab6-ef1f-47a2-bf44-6ae046cc8645" status: 200 len: 1366 microversion: 2.1 time: 0.383006 Aug 27 02:23:05.771776 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 52/105] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:23:05 2022] GET /compute/v2.1/servers/2ad09ab6-ef1f-47a2-bf44-6ae046cc8645 => generated 1366 bytes in 386 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:23:05.773025 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-ce20badf-b893-4e16-8f2e-55d44300fc08 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Calling method '>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:23:05.775546 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-ce20badf-b893-4e16-8f2e-55d44300fc08 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] [instance: 05bed8e9-f530-418a-9819-4f65dabdcda0] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:23:05.793296 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-ce20badf-b893-4e16-8f2e-55d44300fc08 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:23:05.793776 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-ce20badf-b893-4e16-8f2e-55d44300fc08 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:23:05.794206 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-ce20badf-b893-4e16-8f2e-55d44300fc08 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:23:05.794913 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-ce20badf-b893-4e16-8f2e-55d44300fc08 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:23:05.795404 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-ce20badf-b893-4e16-8f2e-55d44300fc08 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:23:05.796066 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-ce20badf-b893-4e16-8f2e-55d44300fc08 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:23:05.967703 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-ce20badf-b893-4e16-8f2e-55d44300fc08 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd945d512ae034ec08838d0a3afde2429', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '880ac4003248441c809a76cfd1b221eb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:23:05.975743 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-ce20badf-b893-4e16-8f2e-55d44300fc08 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] 10.222.0.52 "GET /compute/v2.1/servers/05bed8e9-f530-418a-9819-4f65dabdcda0" status: 200 len: 1322 microversion: 2.1 time: 0.221750 Aug 27 02:23:05.976289 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 54/106] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:23:05 2022] GET /compute/v2.1/servers/05bed8e9-f530-418a-9819-4f65dabdcda0 => generated 1322 bytes in 223 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:23:06.774699 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-e8646d10-3afa-4c28-8270-d883370cc8a4 tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:23:06.775459 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-e8646d10-3afa-4c28-8270-d883370cc8a4 tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] [instance: fbef28bb-3cdf-4a27-8493-add7806af828] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:23:06.781643 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-e8646d10-3afa-4c28-8270-d883370cc8a4 tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:23:06.782124 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-e8646d10-3afa-4c28-8270-d883370cc8a4 tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:23:06.782558 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-e8646d10-3afa-4c28-8270-d883370cc8a4 tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:23:06.783248 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-e8646d10-3afa-4c28-8270-d883370cc8a4 tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:23:06.783699 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-e8646d10-3afa-4c28-8270-d883370cc8a4 tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:23:06.784116 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-e8646d10-3afa-4c28-8270-d883370cc8a4 tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:23:06.797318 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-305b92e6-1b04-4740-b401-ca07813c5a20 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Calling method '>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:23:06.798343 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-305b92e6-1b04-4740-b401-ca07813c5a20 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] [instance: 2ad09ab6-ef1f-47a2-bf44-6ae046cc8645] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:23:06.807431 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-305b92e6-1b04-4740-b401-ca07813c5a20 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:23:06.808226 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-305b92e6-1b04-4740-b401-ca07813c5a20 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:23:06.808897 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-305b92e6-1b04-4740-b401-ca07813c5a20 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:23:06.809911 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-305b92e6-1b04-4740-b401-ca07813c5a20 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:23:06.810588 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-305b92e6-1b04-4740-b401-ca07813c5a20 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:23:06.811255 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-305b92e6-1b04-4740-b401-ca07813c5a20 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:23:06.989864 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-e8646d10-3afa-4c28-8270-d883370cc8a4 tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '55640f750d1b49be9c3f79501a63c1ab', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7ef4b44a3f94d599eacce269b06594c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:23:07.000418 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-e8646d10-3afa-4c28-8270-d883370cc8a4 tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] 10.222.0.52 "GET /compute/v2.1/servers/fbef28bb-3cdf-4a27-8493-add7806af828" status: 200 len: 1314 microversion: 2.1 time: 0.229828 Aug 27 02:23:07.001004 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 53/107] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:23:06 2022] GET /compute/v2.1/servers/fbef28bb-3cdf-4a27-8493-add7806af828 => generated 1314 bytes in 231 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:23:07.006431 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-5c4ab5b8-0f26-4f17-aa9a-4bdf2a9eb203 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:23:07.007092 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-5c4ab5b8-0f26-4f17-aa9a-4bdf2a9eb203 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] [instance: 05bed8e9-f530-418a-9819-4f65dabdcda0] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:23:07.013821 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-5c4ab5b8-0f26-4f17-aa9a-4bdf2a9eb203 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:23:07.015287 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-5c4ab5b8-0f26-4f17-aa9a-4bdf2a9eb203 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:23:07.015964 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-5c4ab5b8-0f26-4f17-aa9a-4bdf2a9eb203 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:23:07.016901 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-5c4ab5b8-0f26-4f17-aa9a-4bdf2a9eb203 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:23:07.017578 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-5c4ab5b8-0f26-4f17-aa9a-4bdf2a9eb203 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:23:07.018223 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-5c4ab5b8-0f26-4f17-aa9a-4bdf2a9eb203 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:23:07.041865 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-305b92e6-1b04-4740-b401-ca07813c5a20 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '26a86351583d49d090b3db09f127d49f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5d5ec8450c0941d18d63558baab54737', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:23:07.056781 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-305b92e6-1b04-4740-b401-ca07813c5a20 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] 10.222.0.52 "GET /compute/v2.1/servers/2ad09ab6-ef1f-47a2-bf44-6ae046cc8645" status: 200 len: 1366 microversion: 2.1 time: 0.264714 Aug 27 02:23:07.058721 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 55/108] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:23:06 2022] GET /compute/v2.1/servers/2ad09ab6-ef1f-47a2-bf44-6ae046cc8645 => generated 1366 bytes in 268 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:23:07.194663 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-5c4ab5b8-0f26-4f17-aa9a-4bdf2a9eb203 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd945d512ae034ec08838d0a3afde2429', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '880ac4003248441c809a76cfd1b221eb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:23:07.203608 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-5c4ab5b8-0f26-4f17-aa9a-4bdf2a9eb203 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] 10.222.0.52 "GET /compute/v2.1/servers/05bed8e9-f530-418a-9819-4f65dabdcda0" status: 200 len: 1322 microversion: 2.1 time: 0.201512 Aug 27 02:23:07.204728 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 54/109] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:23:07 2022] GET /compute/v2.1/servers/05bed8e9-f530-418a-9819-4f65dabdcda0 => generated 1322 bytes in 203 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:23:08.027454 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-8c118cc4-2b61-4950-b26b-335c17de770a tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Calling method '>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:23:08.028082 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-8c118cc4-2b61-4950-b26b-335c17de770a tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] [instance: fbef28bb-3cdf-4a27-8493-add7806af828] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:23:08.035308 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-8c118cc4-2b61-4950-b26b-335c17de770a tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:23:08.036193 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-8c118cc4-2b61-4950-b26b-335c17de770a tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:23:08.036862 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-8c118cc4-2b61-4950-b26b-335c17de770a tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:23:08.038169 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-8c118cc4-2b61-4950-b26b-335c17de770a tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:23:08.039303 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-8c118cc4-2b61-4950-b26b-335c17de770a tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:23:08.040002 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-8c118cc4-2b61-4950-b26b-335c17de770a tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:23:08.088739 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-50ac40c5-462c-41c3-b3c9-b76347d5b749 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:23:08.090353 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-50ac40c5-462c-41c3-b3c9-b76347d5b749 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] [instance: 2ad09ab6-ef1f-47a2-bf44-6ae046cc8645] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:23:08.100463 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-50ac40c5-462c-41c3-b3c9-b76347d5b749 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:23:08.101312 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-50ac40c5-462c-41c3-b3c9-b76347d5b749 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:23:08.102090 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-50ac40c5-462c-41c3-b3c9-b76347d5b749 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:23:08.102831 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-50ac40c5-462c-41c3-b3c9-b76347d5b749 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:23:08.103574 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-50ac40c5-462c-41c3-b3c9-b76347d5b749 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:23:08.104335 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-50ac40c5-462c-41c3-b3c9-b76347d5b749 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:23:08.324159 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-8c118cc4-2b61-4950-b26b-335c17de770a tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '55640f750d1b49be9c3f79501a63c1ab', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7ef4b44a3f94d599eacce269b06594c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:23:08.332938 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-8c118cc4-2b61-4950-b26b-335c17de770a tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] 10.222.0.52 "GET /compute/v2.1/servers/fbef28bb-3cdf-4a27-8493-add7806af828" status: 200 len: 1314 microversion: 2.1 time: 0.309580 Aug 27 02:23:08.337678 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 56/110] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:23:08 2022] GET /compute/v2.1/servers/fbef28bb-3cdf-4a27-8493-add7806af828 => generated 1314 bytes in 315 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:23:08.349570 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-4600ce81-0ef9-449f-86a8-c12ec35003e5 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Calling method '>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:23:08.353332 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-4600ce81-0ef9-449f-86a8-c12ec35003e5 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] [instance: 05bed8e9-f530-418a-9819-4f65dabdcda0] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:23:08.371103 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-4600ce81-0ef9-449f-86a8-c12ec35003e5 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:23:08.377111 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-50ac40c5-462c-41c3-b3c9-b76347d5b749 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '26a86351583d49d090b3db09f127d49f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5d5ec8450c0941d18d63558baab54737', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:23:08.383137 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-4600ce81-0ef9-449f-86a8-c12ec35003e5 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.012s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:23:08.394905 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-4600ce81-0ef9-449f-86a8-c12ec35003e5 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.008s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:23:08.394905 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-4600ce81-0ef9-449f-86a8-c12ec35003e5 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:23:08.394905 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-4600ce81-0ef9-449f-86a8-c12ec35003e5 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:23:08.395122 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-4600ce81-0ef9-449f-86a8-c12ec35003e5 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:23:08.398219 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-50ac40c5-462c-41c3-b3c9-b76347d5b749 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] 10.222.0.52 "GET /compute/v2.1/servers/2ad09ab6-ef1f-47a2-bf44-6ae046cc8645" status: 200 len: 1366 microversion: 2.1 time: 0.308419 Aug 27 02:23:08.400508 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 55/111] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:23:08 2022] GET /compute/v2.1/servers/2ad09ab6-ef1f-47a2-bf44-6ae046cc8645 => generated 1366 bytes in 320 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:23:08.633552 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-4600ce81-0ef9-449f-86a8-c12ec35003e5 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd945d512ae034ec08838d0a3afde2429', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '880ac4003248441c809a76cfd1b221eb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:23:08.644466 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-4600ce81-0ef9-449f-86a8-c12ec35003e5 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] 10.222.0.52 "GET /compute/v2.1/servers/05bed8e9-f530-418a-9819-4f65dabdcda0" status: 200 len: 1322 microversion: 2.1 time: 0.305213 Aug 27 02:23:08.645321 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 57/112] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:23:08 2022] GET /compute/v2.1/servers/05bed8e9-f530-418a-9819-4f65dabdcda0 => generated 1322 bytes in 307 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:23:09.365610 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-a27b99d2-59b7-42d6-84c0-b812966e922d tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:23:09.366584 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-a27b99d2-59b7-42d6-84c0-b812966e922d tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] [instance: fbef28bb-3cdf-4a27-8493-add7806af828] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:23:09.375061 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-a27b99d2-59b7-42d6-84c0-b812966e922d tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:23:09.375663 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-a27b99d2-59b7-42d6-84c0-b812966e922d tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:23:09.376231 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-a27b99d2-59b7-42d6-84c0-b812966e922d tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:23:09.377548 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-a27b99d2-59b7-42d6-84c0-b812966e922d tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:23:09.378625 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-a27b99d2-59b7-42d6-84c0-b812966e922d tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:23:09.379172 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-a27b99d2-59b7-42d6-84c0-b812966e922d tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:23:09.432299 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-c23560ce-1391-4b37-a201-1ff71d4b176b tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Calling method '>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:23:09.433060 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-c23560ce-1391-4b37-a201-1ff71d4b176b tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] [instance: 2ad09ab6-ef1f-47a2-bf44-6ae046cc8645] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:23:09.439457 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-c23560ce-1391-4b37-a201-1ff71d4b176b tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:23:09.439996 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-c23560ce-1391-4b37-a201-1ff71d4b176b tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:23:09.440632 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-c23560ce-1391-4b37-a201-1ff71d4b176b tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:23:09.441628 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-c23560ce-1391-4b37-a201-1ff71d4b176b tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:23:09.442048 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-c23560ce-1391-4b37-a201-1ff71d4b176b tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:23:09.442497 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-c23560ce-1391-4b37-a201-1ff71d4b176b tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:23:09.646746 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-a27b99d2-59b7-42d6-84c0-b812966e922d tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '55640f750d1b49be9c3f79501a63c1ab', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7ef4b44a3f94d599eacce269b06594c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:23:09.658298 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-a27b99d2-59b7-42d6-84c0-b812966e922d tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] 10.222.0.52 "GET /compute/v2.1/servers/fbef28bb-3cdf-4a27-8493-add7806af828" status: 200 len: 1314 microversion: 2.1 time: 0.299666 Aug 27 02:23:09.659249 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 56/113] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:23:09 2022] GET /compute/v2.1/servers/fbef28bb-3cdf-4a27-8493-add7806af828 => generated 1314 bytes in 302 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:23:09.679739 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-28c00a07-01ae-4110-978c-134e1f4ffea7 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:23:09.681686 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-28c00a07-01ae-4110-978c-134e1f4ffea7 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] [instance: 05bed8e9-f530-418a-9819-4f65dabdcda0] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:23:09.683561 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-c23560ce-1391-4b37-a201-1ff71d4b176b tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '26a86351583d49d090b3db09f127d49f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5d5ec8450c0941d18d63558baab54737', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:23:09.686952 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG dbcounter [-] [120864] Writing DB stats nova_cell1:SELECT=100 {{(pid=120864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 27 02:23:09.689036 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-28c00a07-01ae-4110-978c-134e1f4ffea7 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:23:09.689886 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-28c00a07-01ae-4110-978c-134e1f4ffea7 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:23:09.690354 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-28c00a07-01ae-4110-978c-134e1f4ffea7 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:23:09.692780 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-28c00a07-01ae-4110-978c-134e1f4ffea7 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:23:09.692906 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-c23560ce-1391-4b37-a201-1ff71d4b176b tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] 10.222.0.52 "GET /compute/v2.1/servers/2ad09ab6-ef1f-47a2-bf44-6ae046cc8645" status: 200 len: 1366 microversion: 2.1 time: 0.268380 Aug 27 02:23:09.693046 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-28c00a07-01ae-4110-978c-134e1f4ffea7 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:23:09.693480 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 58/114] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:23:09 2022] GET /compute/v2.1/servers/2ad09ab6-ef1f-47a2-bf44-6ae046cc8645 => generated 1366 bytes in 270 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:23:09.697162 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-28c00a07-01ae-4110-978c-134e1f4ffea7 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:23:09.869169 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-28c00a07-01ae-4110-978c-134e1f4ffea7 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd945d512ae034ec08838d0a3afde2429', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '880ac4003248441c809a76cfd1b221eb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:23:09.877559 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-28c00a07-01ae-4110-978c-134e1f4ffea7 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] 10.222.0.52 "GET /compute/v2.1/servers/05bed8e9-f530-418a-9819-4f65dabdcda0" status: 200 len: 1322 microversion: 2.1 time: 0.206001 Aug 27 02:23:09.878235 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 57/115] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:23:09 2022] GET /compute/v2.1/servers/05bed8e9-f530-418a-9819-4f65dabdcda0 => generated 1322 bytes in 208 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:23:10.691953 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-5c931df3-c8ca-4cf2-b865-c3e3f694b37c tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Calling method '>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:23:10.694293 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-5c931df3-c8ca-4cf2-b865-c3e3f694b37c tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] [instance: fbef28bb-3cdf-4a27-8493-add7806af828] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:23:10.705585 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-5c931df3-c8ca-4cf2-b865-c3e3f694b37c tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:23:10.706735 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-5c931df3-c8ca-4cf2-b865-c3e3f694b37c tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:23:10.707466 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-5c931df3-c8ca-4cf2-b865-c3e3f694b37c tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:23:10.710447 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-5c931df3-c8ca-4cf2-b865-c3e3f694b37c tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:23:10.711189 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-5c931df3-c8ca-4cf2-b865-c3e3f694b37c tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:23:10.711950 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-5c931df3-c8ca-4cf2-b865-c3e3f694b37c tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:23:10.720001 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-b4dc0fcb-a4c5-4b11-adfd-6252a5a475f8 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:23:10.721782 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-b4dc0fcb-a4c5-4b11-adfd-6252a5a475f8 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] [instance: 2ad09ab6-ef1f-47a2-bf44-6ae046cc8645] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:23:10.729243 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-b4dc0fcb-a4c5-4b11-adfd-6252a5a475f8 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:23:10.730303 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-b4dc0fcb-a4c5-4b11-adfd-6252a5a475f8 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:23:10.731723 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-b4dc0fcb-a4c5-4b11-adfd-6252a5a475f8 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:23:10.733283 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-b4dc0fcb-a4c5-4b11-adfd-6252a5a475f8 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:23:10.734599 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-b4dc0fcb-a4c5-4b11-adfd-6252a5a475f8 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:23:10.736555 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-b4dc0fcb-a4c5-4b11-adfd-6252a5a475f8 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:23:10.961552 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-5c931df3-c8ca-4cf2-b865-c3e3f694b37c tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '55640f750d1b49be9c3f79501a63c1ab', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7ef4b44a3f94d599eacce269b06594c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:23:10.972768 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-5c931df3-c8ca-4cf2-b865-c3e3f694b37c tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] 10.222.0.52 "GET /compute/v2.1/servers/fbef28bb-3cdf-4a27-8493-add7806af828" status: 200 len: 1314 microversion: 2.1 time: 0.288918 Aug 27 02:23:10.973952 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 59/116] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:23:10 2022] GET /compute/v2.1/servers/fbef28bb-3cdf-4a27-8493-add7806af828 => generated 1314 bytes in 291 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 27 02:23:10.976569 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-b4dc0fcb-a4c5-4b11-adfd-6252a5a475f8 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '26a86351583d49d090b3db09f127d49f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5d5ec8450c0941d18d63558baab54737', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:23:10.980362 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-2da4dbf2-2e60-4c8f-949a-fbf939a20f68 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Calling method '>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:23:10.981197 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-2da4dbf2-2e60-4c8f-949a-fbf939a20f68 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] [instance: 05bed8e9-f530-418a-9819-4f65dabdcda0] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:23:10.987963 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-2da4dbf2-2e60-4c8f-949a-fbf939a20f68 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:23:10.988460 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-2da4dbf2-2e60-4c8f-949a-fbf939a20f68 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:23:10.988833 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-b4dc0fcb-a4c5-4b11-adfd-6252a5a475f8 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] 10.222.0.52 "GET /compute/v2.1/servers/2ad09ab6-ef1f-47a2-bf44-6ae046cc8645" status: 200 len: 1366 microversion: 2.1 time: 0.274985 Aug 27 02:23:10.988994 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-2da4dbf2-2e60-4c8f-949a-fbf939a20f68 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:23:10.989946 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-2da4dbf2-2e60-4c8f-949a-fbf939a20f68 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:23:10.990805 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-2da4dbf2-2e60-4c8f-949a-fbf939a20f68 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:23:10.991235 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-2da4dbf2-2e60-4c8f-949a-fbf939a20f68 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:23:10.992008 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 58/117] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:23:10 2022] GET /compute/v2.1/servers/2ad09ab6-ef1f-47a2-bf44-6ae046cc8645 => generated 1366 bytes in 281 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:23:11.150455 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-2da4dbf2-2e60-4c8f-949a-fbf939a20f68 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd945d512ae034ec08838d0a3afde2429', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '880ac4003248441c809a76cfd1b221eb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:23:11.160247 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-2da4dbf2-2e60-4c8f-949a-fbf939a20f68 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] 10.222.0.52 "GET /compute/v2.1/servers/05bed8e9-f530-418a-9819-4f65dabdcda0" status: 200 len: 1322 microversion: 2.1 time: 0.184852 Aug 27 02:23:11.160833 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 60/118] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:23:10 2022] GET /compute/v2.1/servers/05bed8e9-f530-418a-9819-4f65dabdcda0 => generated 1322 bytes in 186 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:23:11.999889 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-2876422e-8416-4dd6-8635-103b60388b15 tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:23:12.000450 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-2876422e-8416-4dd6-8635-103b60388b15 tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] [instance: fbef28bb-3cdf-4a27-8493-add7806af828] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:23:12.007119 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-2876422e-8416-4dd6-8635-103b60388b15 tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:23:12.007734 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-2876422e-8416-4dd6-8635-103b60388b15 tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:23:12.008205 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-2876422e-8416-4dd6-8635-103b60388b15 tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:23:12.008874 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-2876422e-8416-4dd6-8635-103b60388b15 tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:23:12.009366 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-2876422e-8416-4dd6-8635-103b60388b15 tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:23:12.009816 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-2876422e-8416-4dd6-8635-103b60388b15 tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:23:12.013578 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-d9f1f15f-268d-40fb-aa3f-664bcabb0251 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Calling method '>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:23:12.014627 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-d9f1f15f-268d-40fb-aa3f-664bcabb0251 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] [instance: 2ad09ab6-ef1f-47a2-bf44-6ae046cc8645] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:23:12.021781 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-d9f1f15f-268d-40fb-aa3f-664bcabb0251 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:23:12.022173 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-d9f1f15f-268d-40fb-aa3f-664bcabb0251 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:23:12.022597 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-d9f1f15f-268d-40fb-aa3f-664bcabb0251 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:23:12.023295 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-d9f1f15f-268d-40fb-aa3f-664bcabb0251 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:23:12.023734 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-d9f1f15f-268d-40fb-aa3f-664bcabb0251 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:23:12.024145 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-d9f1f15f-268d-40fb-aa3f-664bcabb0251 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:23:12.227441 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-d9f1f15f-268d-40fb-aa3f-664bcabb0251 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '26a86351583d49d090b3db09f127d49f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5d5ec8450c0941d18d63558baab54737', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:23:12.231478 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-2876422e-8416-4dd6-8635-103b60388b15 tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '55640f750d1b49be9c3f79501a63c1ab', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7ef4b44a3f94d599eacce269b06594c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:23:12.239860 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-d9f1f15f-268d-40fb-aa3f-664bcabb0251 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] 10.222.0.52 "GET /compute/v2.1/servers/2ad09ab6-ef1f-47a2-bf44-6ae046cc8645" status: 200 len: 1366 microversion: 2.1 time: 0.231034 Aug 27 02:23:12.240184 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-2876422e-8416-4dd6-8635-103b60388b15 tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] 10.222.0.52 "GET /compute/v2.1/servers/fbef28bb-3cdf-4a27-8493-add7806af828" status: 200 len: 1314 microversion: 2.1 time: 0.243935 Aug 27 02:23:12.240826 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 61/119] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:23:12 2022] GET /compute/v2.1/servers/2ad09ab6-ef1f-47a2-bf44-6ae046cc8645 => generated 1366 bytes in 233 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:23:12.241023 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 59/120] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:23:11 2022] GET /compute/v2.1/servers/fbef28bb-3cdf-4a27-8493-add7806af828 => generated 1314 bytes in 246 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:23:12.246712 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-126b35e0-5aae-44c3-9b66-83814730b317 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Calling method '>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:23:12.247342 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-126b35e0-5aae-44c3-9b66-83814730b317 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] [instance: 05bed8e9-f530-418a-9819-4f65dabdcda0] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:23:12.253715 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-126b35e0-5aae-44c3-9b66-83814730b317 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:23:12.254196 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-126b35e0-5aae-44c3-9b66-83814730b317 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:23:12.254595 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-126b35e0-5aae-44c3-9b66-83814730b317 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:23:12.255765 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-126b35e0-5aae-44c3-9b66-83814730b317 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:23:12.256302 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-126b35e0-5aae-44c3-9b66-83814730b317 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:23:12.256721 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-126b35e0-5aae-44c3-9b66-83814730b317 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:23:12.405137 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-126b35e0-5aae-44c3-9b66-83814730b317 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd945d512ae034ec08838d0a3afde2429', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '880ac4003248441c809a76cfd1b221eb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:23:12.414718 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-126b35e0-5aae-44c3-9b66-83814730b317 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] 10.222.0.52 "GET /compute/v2.1/servers/05bed8e9-f530-418a-9819-4f65dabdcda0" status: 200 len: 1322 microversion: 2.1 time: 0.171726 Aug 27 02:23:12.415294 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 62/121] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:23:12 2022] GET /compute/v2.1/servers/05bed8e9-f530-418a-9819-4f65dabdcda0 => generated 1322 bytes in 174 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:23:13.262279 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-e4cd66b7-f404-4919-9cc2-7eedf96123b1 tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:23:13.262864 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-e4cd66b7-f404-4919-9cc2-7eedf96123b1 tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] [instance: fbef28bb-3cdf-4a27-8493-add7806af828] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:23:13.269680 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-e4cd66b7-f404-4919-9cc2-7eedf96123b1 tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:23:13.270165 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-e4cd66b7-f404-4919-9cc2-7eedf96123b1 tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:23:13.270593 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-e4cd66b7-f404-4919-9cc2-7eedf96123b1 tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:23:13.271395 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-e4cd66b7-f404-4919-9cc2-7eedf96123b1 tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:23:13.271843 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-e4cd66b7-f404-4919-9cc2-7eedf96123b1 tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:23:13.272268 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-e4cd66b7-f404-4919-9cc2-7eedf96123b1 tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:23:13.272805 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-1049bc14-94e0-49db-9a69-4fb3242c338e tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Calling method '>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:23:13.273437 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-1049bc14-94e0-49db-9a69-4fb3242c338e tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] [instance: 2ad09ab6-ef1f-47a2-bf44-6ae046cc8645] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:23:13.279844 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-1049bc14-94e0-49db-9a69-4fb3242c338e tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:23:13.280325 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-1049bc14-94e0-49db-9a69-4fb3242c338e tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:23:13.280781 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-1049bc14-94e0-49db-9a69-4fb3242c338e tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:23:13.281506 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-1049bc14-94e0-49db-9a69-4fb3242c338e tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:23:13.281951 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-1049bc14-94e0-49db-9a69-4fb3242c338e tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:23:13.282358 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-1049bc14-94e0-49db-9a69-4fb3242c338e tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:23:13.477388 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-e4cd66b7-f404-4919-9cc2-7eedf96123b1 tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '55640f750d1b49be9c3f79501a63c1ab', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7ef4b44a3f94d599eacce269b06594c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:23:13.480239 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-1049bc14-94e0-49db-9a69-4fb3242c338e tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '26a86351583d49d090b3db09f127d49f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5d5ec8450c0941d18d63558baab54737', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:23:13.486424 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-e4cd66b7-f404-4919-9cc2-7eedf96123b1 tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] 10.222.0.52 "GET /compute/v2.1/servers/fbef28bb-3cdf-4a27-8493-add7806af828" status: 200 len: 1314 microversion: 2.1 time: 0.228259 Aug 27 02:23:13.487006 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 60/122] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:23:13 2022] GET /compute/v2.1/servers/fbef28bb-3cdf-4a27-8493-add7806af828 => generated 1314 bytes in 230 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:23:13.488269 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-1049bc14-94e0-49db-9a69-4fb3242c338e tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] 10.222.0.52 "GET /compute/v2.1/servers/2ad09ab6-ef1f-47a2-bf44-6ae046cc8645" status: 200 len: 1366 microversion: 2.1 time: 0.219868 Aug 27 02:23:13.488824 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 63/123] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:23:13 2022] GET /compute/v2.1/servers/2ad09ab6-ef1f-47a2-bf44-6ae046cc8645 => generated 1366 bytes in 221 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:23:13.493488 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-50602c7b-ec4d-42ec-80a8-d4b8f66c395c tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:23:13.493488 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-50602c7b-ec4d-42ec-80a8-d4b8f66c395c tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] [instance: 05bed8e9-f530-418a-9819-4f65dabdcda0] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:23:13.499391 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-50602c7b-ec4d-42ec-80a8-d4b8f66c395c tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:23:13.499867 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-50602c7b-ec4d-42ec-80a8-d4b8f66c395c tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:23:13.500495 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-50602c7b-ec4d-42ec-80a8-d4b8f66c395c tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:23:13.501617 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-50602c7b-ec4d-42ec-80a8-d4b8f66c395c tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:23:13.502225 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-50602c7b-ec4d-42ec-80a8-d4b8f66c395c tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:23:13.502638 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-50602c7b-ec4d-42ec-80a8-d4b8f66c395c tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:23:13.650804 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-50602c7b-ec4d-42ec-80a8-d4b8f66c395c tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd945d512ae034ec08838d0a3afde2429', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '880ac4003248441c809a76cfd1b221eb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:23:13.659296 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-50602c7b-ec4d-42ec-80a8-d4b8f66c395c tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] 10.222.0.52 "GET /compute/v2.1/servers/05bed8e9-f530-418a-9819-4f65dabdcda0" status: 200 len: 1322 microversion: 2.1 time: 0.171256 Aug 27 02:23:13.659969 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 61/124] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:23:13 2022] GET /compute/v2.1/servers/05bed8e9-f530-418a-9819-4f65dabdcda0 => generated 1322 bytes in 172 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:23:14.512366 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-9e74fee3-26c7-495f-a0e5-44ceb7dab007 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Calling method '>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:23:14.513603 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-9e74fee3-26c7-495f-a0e5-44ceb7dab007 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] [instance: 2ad09ab6-ef1f-47a2-bf44-6ae046cc8645] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:23:14.525245 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-9e74fee3-26c7-495f-a0e5-44ceb7dab007 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:23:14.525982 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-9e74fee3-26c7-495f-a0e5-44ceb7dab007 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:23:14.526577 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-9e74fee3-26c7-495f-a0e5-44ceb7dab007 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:23:14.527355 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-a025ba65-67a6-4ef5-9c55-0fc2af4e5450 tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:23:14.527969 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-9e74fee3-26c7-495f-a0e5-44ceb7dab007 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:23:14.528422 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-a025ba65-67a6-4ef5-9c55-0fc2af4e5450 tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] [instance: fbef28bb-3cdf-4a27-8493-add7806af828] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:23:14.528872 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-9e74fee3-26c7-495f-a0e5-44ceb7dab007 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:23:14.529601 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-9e74fee3-26c7-495f-a0e5-44ceb7dab007 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:23:14.552734 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-a025ba65-67a6-4ef5-9c55-0fc2af4e5450 tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:23:14.553822 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-a025ba65-67a6-4ef5-9c55-0fc2af4e5450 tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:23:14.554739 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-a025ba65-67a6-4ef5-9c55-0fc2af4e5450 tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:23:14.555892 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-a025ba65-67a6-4ef5-9c55-0fc2af4e5450 tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:23:14.556684 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-a025ba65-67a6-4ef5-9c55-0fc2af4e5450 tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:23:14.557817 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-a025ba65-67a6-4ef5-9c55-0fc2af4e5450 tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:23:15.307158 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-9e74fee3-26c7-495f-a0e5-44ceb7dab007 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '26a86351583d49d090b3db09f127d49f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5d5ec8450c0941d18d63558baab54737', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:23:15.313040 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-a025ba65-67a6-4ef5-9c55-0fc2af4e5450 tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '55640f750d1b49be9c3f79501a63c1ab', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7ef4b44a3f94d599eacce269b06594c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:23:15.321257 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-a025ba65-67a6-4ef5-9c55-0fc2af4e5450 tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] 10.222.0.52 "GET /compute/v2.1/servers/fbef28bb-3cdf-4a27-8493-add7806af828" status: 200 len: 1314 microversion: 2.1 time: 0.802504 Aug 27 02:23:15.321549 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-9e74fee3-26c7-495f-a0e5-44ceb7dab007 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] 10.222.0.52 "GET /compute/v2.1/servers/2ad09ab6-ef1f-47a2-bf44-6ae046cc8645" status: 200 len: 1366 microversion: 2.1 time: 0.814024 Aug 27 02:23:15.321855 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 62/125] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:23:14 2022] GET /compute/v2.1/servers/fbef28bb-3cdf-4a27-8493-add7806af828 => generated 1314 bytes in 804 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:23:15.326704 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-2f58a72f-9891-4ae9-8922-1b19fc7c7ea2 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:23:15.333772 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-2f58a72f-9891-4ae9-8922-1b19fc7c7ea2 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] [instance: 05bed8e9-f530-418a-9819-4f65dabdcda0] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:23:15.334555 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 64/126] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:23:14 2022] GET /compute/v2.1/servers/2ad09ab6-ef1f-47a2-bf44-6ae046cc8645 => generated 1366 bytes in 828 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:23:15.342887 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [req-f60ae91e-bbab-4933-ae87-0513751c44cf req-920557fa-64c8-4301-b7ac-e1ae75aa45da service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "2ad09ab6-ef1f-47a2-bf44-6ae046cc8645", "tag": "e98d7081-9ec1-49da-9111-4786cc0b96ef"}]} {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 27 02:23:15.356731 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-2f58a72f-9891-4ae9-8922-1b19fc7c7ea2 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:23:15.357267 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-2f58a72f-9891-4ae9-8922-1b19fc7c7ea2 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:23:15.357701 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-2f58a72f-9891-4ae9-8922-1b19fc7c7ea2 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:23:15.358525 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-2f58a72f-9891-4ae9-8922-1b19fc7c7ea2 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:23:15.359066 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-2f58a72f-9891-4ae9-8922-1b19fc7c7ea2 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:23:15.359585 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-2f58a72f-9891-4ae9-8922-1b19fc7c7ea2 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:23:15.361749 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [req-f60ae91e-bbab-4933-ae87-0513751c44cf req-920557fa-64c8-4301-b7ac-e1ae75aa45da service nova] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:23:15.363083 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [req-f60ae91e-bbab-4933-ae87-0513751c44cf req-920557fa-64c8-4301-b7ac-e1ae75aa45da service nova] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:23:15.363834 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [req-f60ae91e-bbab-4933-ae87-0513751c44cf req-920557fa-64c8-4301-b7ac-e1ae75aa45da service nova] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:23:15.440915 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.compute.server_external_events [req-f60ae91e-bbab-4933-ae87-0513751c44cf req-920557fa-64c8-4301-b7ac-e1ae75aa45da service nova] Creating event network-changed:e98d7081-9ec1-49da-9111-4786cc0b96ef for instance 2ad09ab6-ef1f-47a2-bf44-6ae046cc8645 on hv-cinder-84630 Aug 27 02:23:15.443089 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [req-f60ae91e-bbab-4933-ae87-0513751c44cf req-920557fa-64c8-4301-b7ac-e1ae75aa45da service nova] Acquiring lock "compute-rpcapi-router" {{(pid=120864) lock /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:293}} Aug 27 02:23:15.445044 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [req-f60ae91e-bbab-4933-ae87-0513751c44cf req-920557fa-64c8-4301-b7ac-e1ae75aa45da service nova] Acquired lock "compute-rpcapi-router" {{(pid=120864) lock /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:296}} Aug 27 02:23:15.449806 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-40c420cb-1317-4989-868b-8c5f64c0c649 None None] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:23:15.453576 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-40c420cb-1317-4989-868b-8c5f64c0c649 None None] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:23:15.454711 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-40c420cb-1317-4989-868b-8c5f64c0c649 None None] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:23:15.455974 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-40c420cb-1317-4989-868b-8c5f64c0c649 None None] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:23:15.460474 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-40c420cb-1317-4989-868b-8c5f64c0c649 None None] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:23:15.461757 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-40c420cb-1317-4989-868b-8c5f64c0c649 None None] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:23:15.492160 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.compute.rpcapi [None req-40c420cb-1317-4989-868b-8c5f64c0c649 None None] Automatically selected compute RPC version 6.0 from minimum service version 63 Aug 27 02:23:15.495320 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-40c420cb-1317-4989-868b-8c5f64c0c649 None None] Releasing lock "compute-rpcapi-router" {{(pid=120864) lock /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:314}} Aug 27 02:23:15.539696 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-40c420cb-1317-4989-868b-8c5f64c0c649 None None] 10.222.0.52 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.199974 Aug 27 02:23:15.540623 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 65/127] 10.222.0.52 () {64 vars in 1317 bytes} [Sat Aug 27 02:23:15 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 206 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 27 02:23:16.023097 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-2f58a72f-9891-4ae9-8922-1b19fc7c7ea2 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd945d512ae034ec08838d0a3afde2429', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '880ac4003248441c809a76cfd1b221eb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:23:16.040571 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-2f58a72f-9891-4ae9-8922-1b19fc7c7ea2 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] 10.222.0.52 "GET /compute/v2.1/servers/05bed8e9-f530-418a-9819-4f65dabdcda0" status: 200 len: 1322 microversion: 2.1 time: 0.715751 Aug 27 02:23:16.042835 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 63/128] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:23:15 2022] GET /compute/v2.1/servers/05bed8e9-f530-418a-9819-4f65dabdcda0 => generated 1322 bytes in 720 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:23:16.347925 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-9f638644-0645-47ea-9f51-f328c198ac96 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Calling method '>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:23:16.348848 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-9f638644-0645-47ea-9f51-f328c198ac96 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] [instance: 2ad09ab6-ef1f-47a2-bf44-6ae046cc8645] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:23:16.349535 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-a0064777-f538-40f6-954f-066ae2eefa6f tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:23:16.350160 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-a0064777-f538-40f6-954f-066ae2eefa6f tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] [instance: fbef28bb-3cdf-4a27-8493-add7806af828] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:23:16.362584 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-9f638644-0645-47ea-9f51-f328c198ac96 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:23:16.362704 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-a0064777-f538-40f6-954f-066ae2eefa6f tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:23:16.363096 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-a0064777-f538-40f6-954f-066ae2eefa6f tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:23:16.365490 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-9f638644-0645-47ea-9f51-f328c198ac96 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:23:16.366421 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-a0064777-f538-40f6-954f-066ae2eefa6f tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:23:16.367187 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-a0064777-f538-40f6-954f-066ae2eefa6f tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:23:16.367376 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-9f638644-0645-47ea-9f51-f328c198ac96 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:23:16.367664 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-a0064777-f538-40f6-954f-066ae2eefa6f tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:23:16.368095 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-a0064777-f538-40f6-954f-066ae2eefa6f tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:23:16.371748 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-9f638644-0645-47ea-9f51-f328c198ac96 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:23:16.372447 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-9f638644-0645-47ea-9f51-f328c198ac96 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:23:16.374177 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-9f638644-0645-47ea-9f51-f328c198ac96 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:23:17.064583 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-a0064777-f538-40f6-954f-066ae2eefa6f tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '55640f750d1b49be9c3f79501a63c1ab', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7ef4b44a3f94d599eacce269b06594c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:23:17.073012 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-9f638644-0645-47ea-9f51-f328c198ac96 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '26a86351583d49d090b3db09f127d49f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5d5ec8450c0941d18d63558baab54737', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:23:17.082129 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-9f638644-0645-47ea-9f51-f328c198ac96 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] 10.222.0.52 "GET /compute/v2.1/servers/2ad09ab6-ef1f-47a2-bf44-6ae046cc8645" status: 200 len: 1366 microversion: 2.1 time: 0.739446 Aug 27 02:23:17.082891 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 66/129] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:23:16 2022] GET /compute/v2.1/servers/2ad09ab6-ef1f-47a2-bf44-6ae046cc8645 => generated 1366 bytes in 741 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 27 02:23:17.107774 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-a0064777-f538-40f6-954f-066ae2eefa6f tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] 10.222.0.52 "GET /compute/v2.1/servers/fbef28bb-3cdf-4a27-8493-add7806af828" status: 200 len: 1314 microversion: 2.1 time: 0.738174 Aug 27 02:23:17.110395 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 64/130] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:23:16 2022] GET /compute/v2.1/servers/fbef28bb-3cdf-4a27-8493-add7806af828 => generated 1314 bytes in 767 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:23:17.126528 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-d07a2b03-30d3-4d8f-bf56-69ae75a0f99b tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:23:17.132292 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-d07a2b03-30d3-4d8f-bf56-69ae75a0f99b tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] [instance: 05bed8e9-f530-418a-9819-4f65dabdcda0] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:23:17.152687 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-d07a2b03-30d3-4d8f-bf56-69ae75a0f99b tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:23:17.154137 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-d07a2b03-30d3-4d8f-bf56-69ae75a0f99b tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:23:17.154934 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-d07a2b03-30d3-4d8f-bf56-69ae75a0f99b tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:23:17.155655 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-d07a2b03-30d3-4d8f-bf56-69ae75a0f99b tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:23:17.158404 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-d07a2b03-30d3-4d8f-bf56-69ae75a0f99b tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:23:17.158854 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-d07a2b03-30d3-4d8f-bf56-69ae75a0f99b tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:23:17.351764 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [req-305fe688-23e7-480e-8d1c-f292d56a49ea req-e1dfdd6a-2569-44fb-ae0c-ee5c2a632fbf service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "fbef28bb-3cdf-4a27-8493-add7806af828", "tag": "b4168fa6-3b0a-4107-83cf-7d31f29910d4"}]} {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 27 02:23:17.358161 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-d07a2b03-30d3-4d8f-bf56-69ae75a0f99b tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd945d512ae034ec08838d0a3afde2429', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '880ac4003248441c809a76cfd1b221eb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:23:17.365507 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [req-305fe688-23e7-480e-8d1c-f292d56a49ea req-e1dfdd6a-2569-44fb-ae0c-ee5c2a632fbf service nova] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:23:17.366732 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [req-305fe688-23e7-480e-8d1c-f292d56a49ea req-e1dfdd6a-2569-44fb-ae0c-ee5c2a632fbf service nova] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:23:17.369236 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [req-305fe688-23e7-480e-8d1c-f292d56a49ea req-e1dfdd6a-2569-44fb-ae0c-ee5c2a632fbf service nova] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:23:17.369447 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-d07a2b03-30d3-4d8f-bf56-69ae75a0f99b tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] 10.222.0.52 "GET /compute/v2.1/servers/05bed8e9-f530-418a-9819-4f65dabdcda0" status: 200 len: 1322 microversion: 2.1 time: 0.257421 Aug 27 02:23:17.370246 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 65/131] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:23:17 2022] GET /compute/v2.1/servers/05bed8e9-f530-418a-9819-4f65dabdcda0 => generated 1322 bytes in 259 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:23:17.391786 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.compute.server_external_events [req-305fe688-23e7-480e-8d1c-f292d56a49ea req-e1dfdd6a-2569-44fb-ae0c-ee5c2a632fbf service nova] Creating event network-changed:b4168fa6-3b0a-4107-83cf-7d31f29910d4 for instance fbef28bb-3cdf-4a27-8493-add7806af828 on hv-cinder-84630 Aug 27 02:23:17.402473 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [req-305fe688-23e7-480e-8d1c-f292d56a49ea req-e1dfdd6a-2569-44fb-ae0c-ee5c2a632fbf service nova] 10.222.0.52 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.318356 Aug 27 02:23:17.403368 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 67/132] 10.222.0.52 () {64 vars in 1317 bytes} [Sat Aug 27 02:23:17 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 320 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 27 02:23:18.106870 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-15d30e5a-cee8-41e3-9bba-77060e1d8107 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:23:18.107509 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-15d30e5a-cee8-41e3-9bba-77060e1d8107 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] [instance: 2ad09ab6-ef1f-47a2-bf44-6ae046cc8645] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:23:18.114133 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-15d30e5a-cee8-41e3-9bba-77060e1d8107 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:23:18.114777 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-15d30e5a-cee8-41e3-9bba-77060e1d8107 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:23:18.115344 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-15d30e5a-cee8-41e3-9bba-77060e1d8107 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:23:18.116661 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-15d30e5a-cee8-41e3-9bba-77060e1d8107 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:23:18.116661 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-15d30e5a-cee8-41e3-9bba-77060e1d8107 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:23:18.116912 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-15d30e5a-cee8-41e3-9bba-77060e1d8107 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:23:18.136710 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-fd42a676-a32b-4e89-a699-2c277483d520 tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Calling method '>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:23:18.137636 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-fd42a676-a32b-4e89-a699-2c277483d520 tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] [instance: fbef28bb-3cdf-4a27-8493-add7806af828] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:23:18.144821 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-fd42a676-a32b-4e89-a699-2c277483d520 tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:23:18.145369 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-fd42a676-a32b-4e89-a699-2c277483d520 tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:23:18.145840 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-fd42a676-a32b-4e89-a699-2c277483d520 tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:23:18.146720 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-fd42a676-a32b-4e89-a699-2c277483d520 tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:23:18.147504 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-fd42a676-a32b-4e89-a699-2c277483d520 tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:23:18.148078 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-fd42a676-a32b-4e89-a699-2c277483d520 tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:23:18.316087 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-15d30e5a-cee8-41e3-9bba-77060e1d8107 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '26a86351583d49d090b3db09f127d49f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5d5ec8450c0941d18d63558baab54737', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:23:18.340139 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-15d30e5a-cee8-41e3-9bba-77060e1d8107 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] 10.222.0.52 "GET /compute/v2.1/servers/2ad09ab6-ef1f-47a2-bf44-6ae046cc8645" status: 200 len: 1366 microversion: 2.1 time: 0.236720 Aug 27 02:23:18.355085 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 66/133] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:23:18 2022] GET /compute/v2.1/servers/2ad09ab6-ef1f-47a2-bf44-6ae046cc8645 => generated 1366 bytes in 240 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:23:18.391107 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-fd42a676-a32b-4e89-a699-2c277483d520 tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '55640f750d1b49be9c3f79501a63c1ab', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7ef4b44a3f94d599eacce269b06594c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:23:18.400428 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-ff0098f2-dcf2-4e57-a4c5-dd30566ce5e9 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:23:18.400988 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-ff0098f2-dcf2-4e57-a4c5-dd30566ce5e9 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] [instance: 05bed8e9-f530-418a-9819-4f65dabdcda0] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:23:18.411423 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-ff0098f2-dcf2-4e57-a4c5-dd30566ce5e9 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:23:18.411843 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-ff0098f2-dcf2-4e57-a4c5-dd30566ce5e9 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:23:18.412455 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-ff0098f2-dcf2-4e57-a4c5-dd30566ce5e9 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:23:18.415916 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-ff0098f2-dcf2-4e57-a4c5-dd30566ce5e9 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:23:18.415916 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-ff0098f2-dcf2-4e57-a4c5-dd30566ce5e9 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:23:18.415916 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-ff0098f2-dcf2-4e57-a4c5-dd30566ce5e9 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:23:18.429952 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-fd42a676-a32b-4e89-a699-2c277483d520 tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] 10.222.0.52 "GET /compute/v2.1/servers/fbef28bb-3cdf-4a27-8493-add7806af828" status: 200 len: 1314 microversion: 2.1 time: 0.297193 Aug 27 02:23:18.449500 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 68/134] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:23:18 2022] GET /compute/v2.1/servers/fbef28bb-3cdf-4a27-8493-add7806af828 => generated 1314 bytes in 305 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 27 02:23:18.449818 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG dbcounter [-] [120863] Writing DB stats nova_cell1:SELECT=100 {{(pid=120863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 27 02:23:18.581094 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-ff0098f2-dcf2-4e57-a4c5-dd30566ce5e9 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd945d512ae034ec08838d0a3afde2429', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '880ac4003248441c809a76cfd1b221eb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:23:18.589961 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-ff0098f2-dcf2-4e57-a4c5-dd30566ce5e9 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] 10.222.0.52 "GET /compute/v2.1/servers/05bed8e9-f530-418a-9819-4f65dabdcda0" status: 200 len: 1322 microversion: 2.1 time: 0.198483 Aug 27 02:23:18.590757 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 67/135] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:23:18 2022] GET /compute/v2.1/servers/05bed8e9-f530-418a-9819-4f65dabdcda0 => generated 1322 bytes in 203 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:23:19.365723 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-efd3ad89-2723-4ad4-a84b-21db288f1ca7 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Calling method '>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:23:19.366318 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-efd3ad89-2723-4ad4-a84b-21db288f1ca7 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] [instance: 2ad09ab6-ef1f-47a2-bf44-6ae046cc8645] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:23:19.375985 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-efd3ad89-2723-4ad4-a84b-21db288f1ca7 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:23:19.375985 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-efd3ad89-2723-4ad4-a84b-21db288f1ca7 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:23:19.376186 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-efd3ad89-2723-4ad4-a84b-21db288f1ca7 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:23:19.376782 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-efd3ad89-2723-4ad4-a84b-21db288f1ca7 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:23:19.378114 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-efd3ad89-2723-4ad4-a84b-21db288f1ca7 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:23:19.379079 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-efd3ad89-2723-4ad4-a84b-21db288f1ca7 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:23:19.426327 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [req-cb964a17-0bca-45a1-8be1-d1fa81246ae3 req-2405ecfd-e057-4e70-bc3e-e5c58f3b3653 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "e2bf7024-0779-4e08-844a-43b78187e770", "tag": "66f55f99-dc75-425b-9214-a2edc384e389"}]} {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 27 02:23:19.437388 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [req-cb964a17-0bca-45a1-8be1-d1fa81246ae3 req-2405ecfd-e057-4e70-bc3e-e5c58f3b3653 service nova] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:23:19.437866 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [req-cb964a17-0bca-45a1-8be1-d1fa81246ae3 req-2405ecfd-e057-4e70-bc3e-e5c58f3b3653 service nova] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:23:19.438284 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [req-cb964a17-0bca-45a1-8be1-d1fa81246ae3 req-2405ecfd-e057-4e70-bc3e-e5c58f3b3653 service nova] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:23:19.453195 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.compute.server_external_events [req-cb964a17-0bca-45a1-8be1-d1fa81246ae3 req-2405ecfd-e057-4e70-bc3e-e5c58f3b3653 service nova] Creating event network-changed:66f55f99-dc75-425b-9214-a2edc384e389 for instance e2bf7024-0779-4e08-844a-43b78187e770 on hv-cinder-84630 Aug 27 02:23:19.460014 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [req-cb964a17-0bca-45a1-8be1-d1fa81246ae3 req-2405ecfd-e057-4e70-bc3e-e5c58f3b3653 service nova] 10.222.0.52 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.038256 Aug 27 02:23:19.460870 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 68/136] 10.222.0.52 () {64 vars in 1317 bytes} [Sat Aug 27 02:23:19 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 40 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 27 02:23:19.466260 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-c9e418ed-1dbc-4055-b8c4-8a32ccafda13 tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:23:19.467008 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-c9e418ed-1dbc-4055-b8c4-8a32ccafda13 tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] [instance: fbef28bb-3cdf-4a27-8493-add7806af828] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:23:19.473679 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-c9e418ed-1dbc-4055-b8c4-8a32ccafda13 tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:23:19.474295 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-c9e418ed-1dbc-4055-b8c4-8a32ccafda13 tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:23:19.475085 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-c9e418ed-1dbc-4055-b8c4-8a32ccafda13 tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:23:19.475855 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-c9e418ed-1dbc-4055-b8c4-8a32ccafda13 tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:23:19.476524 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-c9e418ed-1dbc-4055-b8c4-8a32ccafda13 tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:23:19.476899 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-c9e418ed-1dbc-4055-b8c4-8a32ccafda13 tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:23:19.694207 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-efd3ad89-2723-4ad4-a84b-21db288f1ca7 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '26a86351583d49d090b3db09f127d49f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5d5ec8450c0941d18d63558baab54737', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:23:19.704768 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-efd3ad89-2723-4ad4-a84b-21db288f1ca7 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] 10.222.0.52 "GET /compute/v2.1/servers/2ad09ab6-ef1f-47a2-bf44-6ae046cc8645" status: 200 len: 1366 microversion: 2.1 time: 0.345272 Aug 27 02:23:19.705495 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 69/137] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:23:19 2022] GET /compute/v2.1/servers/2ad09ab6-ef1f-47a2-bf44-6ae046cc8645 => generated 1366 bytes in 347 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:23:19.722920 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-583ef038-cd39-49cf-95b9-dd101c6a826d tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Calling method '>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:23:19.723529 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-583ef038-cd39-49cf-95b9-dd101c6a826d tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] [instance: 05bed8e9-f530-418a-9819-4f65dabdcda0] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:23:19.734849 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-583ef038-cd39-49cf-95b9-dd101c6a826d tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:23:19.735344 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-583ef038-cd39-49cf-95b9-dd101c6a826d tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:23:19.735784 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-583ef038-cd39-49cf-95b9-dd101c6a826d tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:23:19.736484 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-583ef038-cd39-49cf-95b9-dd101c6a826d tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:23:19.736933 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-583ef038-cd39-49cf-95b9-dd101c6a826d tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:23:19.737385 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-583ef038-cd39-49cf-95b9-dd101c6a826d tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:23:19.780754 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-c9e418ed-1dbc-4055-b8c4-8a32ccafda13 tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '55640f750d1b49be9c3f79501a63c1ab', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7ef4b44a3f94d599eacce269b06594c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:23:19.789962 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-c9e418ed-1dbc-4055-b8c4-8a32ccafda13 tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] 10.222.0.52 "GET /compute/v2.1/servers/fbef28bb-3cdf-4a27-8493-add7806af828" status: 200 len: 1314 microversion: 2.1 time: 0.327791 Aug 27 02:23:19.790657 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 69/138] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:23:19 2022] GET /compute/v2.1/servers/fbef28bb-3cdf-4a27-8493-add7806af828 => generated 1314 bytes in 329 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:23:19.935193 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-583ef038-cd39-49cf-95b9-dd101c6a826d tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd945d512ae034ec08838d0a3afde2429', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '880ac4003248441c809a76cfd1b221eb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:23:19.943166 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-583ef038-cd39-49cf-95b9-dd101c6a826d tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] 10.222.0.52 "GET /compute/v2.1/servers/05bed8e9-f530-418a-9819-4f65dabdcda0" status: 200 len: 1322 microversion: 2.1 time: 0.236468 Aug 27 02:23:19.957456 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 70/139] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:23:19 2022] GET /compute/v2.1/servers/05bed8e9-f530-418a-9819-4f65dabdcda0 => generated 1322 bytes in 251 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:23:20.733181 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-984b2eb3-531f-431a-b872-c5cb3c84368d tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:23:20.733944 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-984b2eb3-531f-431a-b872-c5cb3c84368d tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] [instance: 2ad09ab6-ef1f-47a2-bf44-6ae046cc8645] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:23:20.740954 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-984b2eb3-531f-431a-b872-c5cb3c84368d tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:23:20.741617 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-984b2eb3-531f-431a-b872-c5cb3c84368d tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:23:20.742317 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-984b2eb3-531f-431a-b872-c5cb3c84368d tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:23:20.743379 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-984b2eb3-531f-431a-b872-c5cb3c84368d tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:23:20.744013 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-984b2eb3-531f-431a-b872-c5cb3c84368d tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:23:20.745224 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-984b2eb3-531f-431a-b872-c5cb3c84368d tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:23:20.814226 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-9108a4ba-d998-4a88-9c0f-a4332dd68d81 tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Calling method '>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:23:20.814930 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-9108a4ba-d998-4a88-9c0f-a4332dd68d81 tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] [instance: fbef28bb-3cdf-4a27-8493-add7806af828] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:23:20.830413 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-9108a4ba-d998-4a88-9c0f-a4332dd68d81 tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:23:20.831844 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-9108a4ba-d998-4a88-9c0f-a4332dd68d81 tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:23:20.832529 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-9108a4ba-d998-4a88-9c0f-a4332dd68d81 tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:23:20.836056 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-9108a4ba-d998-4a88-9c0f-a4332dd68d81 tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:23:20.836599 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-9108a4ba-d998-4a88-9c0f-a4332dd68d81 tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:23:20.837080 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-9108a4ba-d998-4a88-9c0f-a4332dd68d81 tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:23:21.044225 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-984b2eb3-531f-431a-b872-c5cb3c84368d tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '26a86351583d49d090b3db09f127d49f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5d5ec8450c0941d18d63558baab54737', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:23:21.062726 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-984b2eb3-531f-431a-b872-c5cb3c84368d tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] 10.222.0.52 "GET /compute/v2.1/servers/2ad09ab6-ef1f-47a2-bf44-6ae046cc8645" status: 200 len: 1366 microversion: 2.1 time: 0.339090 Aug 27 02:23:21.063960 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 70/140] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:23:20 2022] GET /compute/v2.1/servers/2ad09ab6-ef1f-47a2-bf44-6ae046cc8645 => generated 1366 bytes in 341 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:23:21.072030 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-0f9b5e40-0177-4e22-af7e-68554d6650d2 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:23:21.073015 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-0f9b5e40-0177-4e22-af7e-68554d6650d2 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] [instance: 05bed8e9-f530-418a-9819-4f65dabdcda0] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:23:21.081617 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-0f9b5e40-0177-4e22-af7e-68554d6650d2 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:23:21.082357 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-0f9b5e40-0177-4e22-af7e-68554d6650d2 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:23:21.082933 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-0f9b5e40-0177-4e22-af7e-68554d6650d2 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:23:21.083905 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-0f9b5e40-0177-4e22-af7e-68554d6650d2 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:23:21.084512 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-0f9b5e40-0177-4e22-af7e-68554d6650d2 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:23:21.086426 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-0f9b5e40-0177-4e22-af7e-68554d6650d2 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:23:21.088023 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-9108a4ba-d998-4a88-9c0f-a4332dd68d81 tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '55640f750d1b49be9c3f79501a63c1ab', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7ef4b44a3f94d599eacce269b06594c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:23:21.096383 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-9108a4ba-d998-4a88-9c0f-a4332dd68d81 tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] 10.222.0.52 "GET /compute/v2.1/servers/fbef28bb-3cdf-4a27-8493-add7806af828" status: 200 len: 1314 microversion: 2.1 time: 0.288886 Aug 27 02:23:21.097340 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 71/141] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:23:20 2022] GET /compute/v2.1/servers/fbef28bb-3cdf-4a27-8493-add7806af828 => generated 1314 bytes in 291 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 27 02:23:21.436172 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-0f9b5e40-0177-4e22-af7e-68554d6650d2 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd945d512ae034ec08838d0a3afde2429', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '880ac4003248441c809a76cfd1b221eb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:23:21.444213 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-0f9b5e40-0177-4e22-af7e-68554d6650d2 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] 10.222.0.52 "GET /compute/v2.1/servers/05bed8e9-f530-418a-9819-4f65dabdcda0" status: 200 len: 1322 microversion: 2.1 time: 0.379097 Aug 27 02:23:21.444803 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 71/142] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:23:21 2022] GET /compute/v2.1/servers/05bed8e9-f530-418a-9819-4f65dabdcda0 => generated 1322 bytes in 380 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:23:22.090507 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-7270d7ae-df15-4cde-b9b9-77c48216901d tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Calling method '>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:23:22.091991 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-7270d7ae-df15-4cde-b9b9-77c48216901d tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] [instance: 2ad09ab6-ef1f-47a2-bf44-6ae046cc8645] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:23:22.105306 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-7270d7ae-df15-4cde-b9b9-77c48216901d tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:23:22.105826 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-7270d7ae-df15-4cde-b9b9-77c48216901d tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:23:22.106294 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-7270d7ae-df15-4cde-b9b9-77c48216901d tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:23:22.107142 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-7270d7ae-df15-4cde-b9b9-77c48216901d tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:23:22.107693 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-7270d7ae-df15-4cde-b9b9-77c48216901d tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:23:22.108281 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-7270d7ae-df15-4cde-b9b9-77c48216901d tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:23:22.125882 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-d19a0338-ded2-4671-b970-49b9326d44f3 tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:23:22.128060 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-d19a0338-ded2-4671-b970-49b9326d44f3 tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] [instance: fbef28bb-3cdf-4a27-8493-add7806af828] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:23:22.140757 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG dbcounter [-] [120863] Writing DB stats nova_api:SELECT=71 {{(pid=120863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 27 02:23:22.144010 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-d19a0338-ded2-4671-b970-49b9326d44f3 tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:23:22.145073 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-d19a0338-ded2-4671-b970-49b9326d44f3 tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:23:22.147094 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-d19a0338-ded2-4671-b970-49b9326d44f3 tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:23:22.148461 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-d19a0338-ded2-4671-b970-49b9326d44f3 tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:23:22.149477 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-d19a0338-ded2-4671-b970-49b9326d44f3 tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:23:22.150434 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-d19a0338-ded2-4671-b970-49b9326d44f3 tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:23:22.335612 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-7270d7ae-df15-4cde-b9b9-77c48216901d tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '26a86351583d49d090b3db09f127d49f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5d5ec8450c0941d18d63558baab54737', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:23:22.343778 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-7270d7ae-df15-4cde-b9b9-77c48216901d tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] 10.222.0.52 "GET /compute/v2.1/servers/2ad09ab6-ef1f-47a2-bf44-6ae046cc8645" status: 200 len: 1366 microversion: 2.1 time: 0.261113 Aug 27 02:23:22.344371 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 72/143] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:23:22 2022] GET /compute/v2.1/servers/2ad09ab6-ef1f-47a2-bf44-6ae046cc8645 => generated 1366 bytes in 263 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:23:22.366230 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-d19a0338-ded2-4671-b970-49b9326d44f3 tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '55640f750d1b49be9c3f79501a63c1ab', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7ef4b44a3f94d599eacce269b06594c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:23:22.373962 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-d19a0338-ded2-4671-b970-49b9326d44f3 tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] 10.222.0.52 "GET /compute/v2.1/servers/fbef28bb-3cdf-4a27-8493-add7806af828" status: 200 len: 1314 microversion: 2.1 time: 0.257191 Aug 27 02:23:22.374908 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 72/144] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:23:22 2022] GET /compute/v2.1/servers/fbef28bb-3cdf-4a27-8493-add7806af828 => generated 1314 bytes in 259 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:23:22.466898 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-d6f8bc0b-b832-49e9-91a9-c8e694295fac tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Calling method '>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:23:22.467503 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-d6f8bc0b-b832-49e9-91a9-c8e694295fac tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] [instance: 05bed8e9-f530-418a-9819-4f65dabdcda0] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:23:22.475210 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-d6f8bc0b-b832-49e9-91a9-c8e694295fac tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:23:22.475616 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-d6f8bc0b-b832-49e9-91a9-c8e694295fac tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:23:22.476751 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-d6f8bc0b-b832-49e9-91a9-c8e694295fac tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:23:22.478539 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-d6f8bc0b-b832-49e9-91a9-c8e694295fac tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:23:22.479177 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-d6f8bc0b-b832-49e9-91a9-c8e694295fac tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:23:22.480098 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-d6f8bc0b-b832-49e9-91a9-c8e694295fac tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:23:22.621938 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-d6f8bc0b-b832-49e9-91a9-c8e694295fac tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd945d512ae034ec08838d0a3afde2429', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '880ac4003248441c809a76cfd1b221eb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:23:22.631823 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-d6f8bc0b-b832-49e9-91a9-c8e694295fac tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] 10.222.0.52 "GET /compute/v2.1/servers/05bed8e9-f530-418a-9819-4f65dabdcda0" status: 200 len: 1322 microversion: 2.1 time: 0.168892 Aug 27 02:23:22.632409 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 73/145] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:23:22 2022] GET /compute/v2.1/servers/05bed8e9-f530-418a-9819-4f65dabdcda0 => generated 1322 bytes in 170 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:23:23.364470 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-7ec173f7-3e34-4019-b8a9-f48f09d86f94 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:23:23.365308 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-7ec173f7-3e34-4019-b8a9-f48f09d86f94 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] [instance: 2ad09ab6-ef1f-47a2-bf44-6ae046cc8645] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:23:23.371683 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-7ec173f7-3e34-4019-b8a9-f48f09d86f94 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:23:23.372187 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-7ec173f7-3e34-4019-b8a9-f48f09d86f94 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:23:23.372599 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-7ec173f7-3e34-4019-b8a9-f48f09d86f94 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:23:23.373350 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-7ec173f7-3e34-4019-b8a9-f48f09d86f94 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:23:23.373769 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-7ec173f7-3e34-4019-b8a9-f48f09d86f94 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:23:23.374189 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-7ec173f7-3e34-4019-b8a9-f48f09d86f94 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:23:23.396245 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-9b6c30e4-4dd0-4291-ba44-014995020156 tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Calling method '>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:23:23.396859 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-9b6c30e4-4dd0-4291-ba44-014995020156 tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] [instance: fbef28bb-3cdf-4a27-8493-add7806af828] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:23:23.403594 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-9b6c30e4-4dd0-4291-ba44-014995020156 tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:23:23.404330 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-9b6c30e4-4dd0-4291-ba44-014995020156 tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:23:23.404935 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-9b6c30e4-4dd0-4291-ba44-014995020156 tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:23:23.405905 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-9b6c30e4-4dd0-4291-ba44-014995020156 tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:23:23.406334 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-9b6c30e4-4dd0-4291-ba44-014995020156 tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:23:23.406925 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-9b6c30e4-4dd0-4291-ba44-014995020156 tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:23:23.574582 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-7ec173f7-3e34-4019-b8a9-f48f09d86f94 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '26a86351583d49d090b3db09f127d49f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5d5ec8450c0941d18d63558baab54737', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:23:23.582929 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-7ec173f7-3e34-4019-b8a9-f48f09d86f94 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] 10.222.0.52 "GET /compute/v2.1/servers/2ad09ab6-ef1f-47a2-bf44-6ae046cc8645" status: 200 len: 1366 microversion: 2.1 time: 0.222562 Aug 27 02:23:23.583578 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 73/146] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:23:23 2022] GET /compute/v2.1/servers/2ad09ab6-ef1f-47a2-bf44-6ae046cc8645 => generated 1366 bytes in 224 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:23:23.603866 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-9b6c30e4-4dd0-4291-ba44-014995020156 tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '55640f750d1b49be9c3f79501a63c1ab', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7ef4b44a3f94d599eacce269b06594c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:23:23.612069 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-9b6c30e4-4dd0-4291-ba44-014995020156 tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] 10.222.0.52 "GET /compute/v2.1/servers/fbef28bb-3cdf-4a27-8493-add7806af828" status: 200 len: 1314 microversion: 2.1 time: 0.219967 Aug 27 02:23:23.612638 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 74/147] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:23:23 2022] GET /compute/v2.1/servers/fbef28bb-3cdf-4a27-8493-add7806af828 => generated 1314 bytes in 221 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:23:23.652722 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-f0cd285e-f3ff-46b6-a6e8-dabcb093fa33 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:23:23.653344 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-f0cd285e-f3ff-46b6-a6e8-dabcb093fa33 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] [instance: 05bed8e9-f530-418a-9819-4f65dabdcda0] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:23:23.659892 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-f0cd285e-f3ff-46b6-a6e8-dabcb093fa33 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:23:23.660520 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-f0cd285e-f3ff-46b6-a6e8-dabcb093fa33 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:23:23.661089 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-f0cd285e-f3ff-46b6-a6e8-dabcb093fa33 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:23:23.662772 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-f0cd285e-f3ff-46b6-a6e8-dabcb093fa33 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:23:23.662772 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-f0cd285e-f3ff-46b6-a6e8-dabcb093fa33 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:23:23.663196 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-f0cd285e-f3ff-46b6-a6e8-dabcb093fa33 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:23:23.804345 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-f0cd285e-f3ff-46b6-a6e8-dabcb093fa33 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd945d512ae034ec08838d0a3afde2429', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '880ac4003248441c809a76cfd1b221eb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:23:23.812345 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-f0cd285e-f3ff-46b6-a6e8-dabcb093fa33 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] 10.222.0.52 "GET /compute/v2.1/servers/05bed8e9-f530-418a-9819-4f65dabdcda0" status: 200 len: 1322 microversion: 2.1 time: 0.163259 Aug 27 02:23:23.812829 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 74/148] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:23:23 2022] GET /compute/v2.1/servers/05bed8e9-f530-418a-9819-4f65dabdcda0 => generated 1322 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:23:24.605462 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-5db1a637-4d43-42db-b757-0ed8a76d19dd tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Calling method '>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:23:24.606218 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-5db1a637-4d43-42db-b757-0ed8a76d19dd tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] [instance: 2ad09ab6-ef1f-47a2-bf44-6ae046cc8645] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:23:24.613513 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-5db1a637-4d43-42db-b757-0ed8a76d19dd tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:23:24.614000 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-5db1a637-4d43-42db-b757-0ed8a76d19dd tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:23:24.614712 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-5db1a637-4d43-42db-b757-0ed8a76d19dd tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:23:24.615455 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-5db1a637-4d43-42db-b757-0ed8a76d19dd tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:23:24.615902 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-5db1a637-4d43-42db-b757-0ed8a76d19dd tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:23:24.616376 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-5db1a637-4d43-42db-b757-0ed8a76d19dd tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:23:24.635282 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-6333dbee-62c3-4a65-98b1-da6076e0d576 tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:23:24.636141 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-6333dbee-62c3-4a65-98b1-da6076e0d576 tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] [instance: fbef28bb-3cdf-4a27-8493-add7806af828] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:23:24.642966 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-6333dbee-62c3-4a65-98b1-da6076e0d576 tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:23:24.643555 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-6333dbee-62c3-4a65-98b1-da6076e0d576 tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:23:24.643910 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-6333dbee-62c3-4a65-98b1-da6076e0d576 tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:23:24.644676 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-6333dbee-62c3-4a65-98b1-da6076e0d576 tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:23:24.645145 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-6333dbee-62c3-4a65-98b1-da6076e0d576 tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:23:24.646323 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-6333dbee-62c3-4a65-98b1-da6076e0d576 tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:23:24.837911 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-5db1a637-4d43-42db-b757-0ed8a76d19dd tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '26a86351583d49d090b3db09f127d49f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5d5ec8450c0941d18d63558baab54737', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:23:24.844312 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-6333dbee-62c3-4a65-98b1-da6076e0d576 tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '55640f750d1b49be9c3f79501a63c1ab', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7ef4b44a3f94d599eacce269b06594c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:23:24.850460 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-5db1a637-4d43-42db-b757-0ed8a76d19dd tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] 10.222.0.52 "GET /compute/v2.1/servers/2ad09ab6-ef1f-47a2-bf44-6ae046cc8645" status: 200 len: 1366 microversion: 2.1 time: 0.249275 Aug 27 02:23:24.855667 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-6333dbee-62c3-4a65-98b1-da6076e0d576 tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] 10.222.0.52 "GET /compute/v2.1/servers/fbef28bb-3cdf-4a27-8493-add7806af828" status: 200 len: 1314 microversion: 2.1 time: 0.224910 Aug 27 02:23:24.857423 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 75/149] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:23:24 2022] GET /compute/v2.1/servers/fbef28bb-3cdf-4a27-8493-add7806af828 => generated 1314 bytes in 228 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 27 02:23:24.862597 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-f4aa8836-5362-4342-8b73-6aa187eb0c7e tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:23:24.863502 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-f4aa8836-5362-4342-8b73-6aa187eb0c7e tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] [instance: 05bed8e9-f530-418a-9819-4f65dabdcda0] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:23:24.864441 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 75/150] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:23:24 2022] GET /compute/v2.1/servers/2ad09ab6-ef1f-47a2-bf44-6ae046cc8645 => generated 1366 bytes in 264 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:23:24.871327 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-f4aa8836-5362-4342-8b73-6aa187eb0c7e tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:23:24.872612 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-f4aa8836-5362-4342-8b73-6aa187eb0c7e tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:23:24.873023 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-f4aa8836-5362-4342-8b73-6aa187eb0c7e tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:23:24.873990 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-f4aa8836-5362-4342-8b73-6aa187eb0c7e tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:23:24.889555 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-f4aa8836-5362-4342-8b73-6aa187eb0c7e tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.012s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:23:24.890102 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-f4aa8836-5362-4342-8b73-6aa187eb0c7e tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:23:25.490097 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-f4aa8836-5362-4342-8b73-6aa187eb0c7e tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd945d512ae034ec08838d0a3afde2429', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '880ac4003248441c809a76cfd1b221eb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:23:25.504661 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-f4aa8836-5362-4342-8b73-6aa187eb0c7e tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] 10.222.0.52 "GET /compute/v2.1/servers/05bed8e9-f530-418a-9819-4f65dabdcda0" status: 200 len: 1322 microversion: 2.1 time: 0.644927 Aug 27 02:23:25.507276 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 76/151] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:23:24 2022] GET /compute/v2.1/servers/05bed8e9-f530-418a-9819-4f65dabdcda0 => generated 1322 bytes in 649 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:23:25.872302 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-9d43d3a5-86e9-46d1-861b-dc5f7e55b27d tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Calling method '>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:23:25.872981 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-9d43d3a5-86e9-46d1-861b-dc5f7e55b27d tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] [instance: 2ad09ab6-ef1f-47a2-bf44-6ae046cc8645] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:23:25.880102 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-5d5f743b-c7cd-438e-84e6-a240688704df tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:23:25.880877 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-5d5f743b-c7cd-438e-84e6-a240688704df tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] [instance: fbef28bb-3cdf-4a27-8493-add7806af828] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:23:25.884821 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG dbcounter [-] [120864] Writing DB stats nova_cell0:SELECT=2 {{(pid=120864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 27 02:23:25.888368 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-9d43d3a5-86e9-46d1-861b-dc5f7e55b27d tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:23:25.888845 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-9d43d3a5-86e9-46d1-861b-dc5f7e55b27d tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:23:25.889287 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-9d43d3a5-86e9-46d1-861b-dc5f7e55b27d tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:23:25.890326 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-9d43d3a5-86e9-46d1-861b-dc5f7e55b27d tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:23:25.891085 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-9d43d3a5-86e9-46d1-861b-dc5f7e55b27d tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:23:25.892144 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-9d43d3a5-86e9-46d1-861b-dc5f7e55b27d tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:23:25.899086 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-5d5f743b-c7cd-438e-84e6-a240688704df tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:23:25.899768 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-5d5f743b-c7cd-438e-84e6-a240688704df tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:23:25.900321 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-5d5f743b-c7cd-438e-84e6-a240688704df tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:23:25.901142 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-5d5f743b-c7cd-438e-84e6-a240688704df tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:23:25.901597 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-5d5f743b-c7cd-438e-84e6-a240688704df tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:23:25.902110 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-5d5f743b-c7cd-438e-84e6-a240688704df tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:23:25.920099 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG dbcounter [-] [120864] Writing DB stats nova_cell1:SELECT=100 {{(pid=120864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 27 02:23:26.143885 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-9d43d3a5-86e9-46d1-861b-dc5f7e55b27d tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '26a86351583d49d090b3db09f127d49f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5d5ec8450c0941d18d63558baab54737', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:23:26.154794 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-5d5f743b-c7cd-438e-84e6-a240688704df tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '55640f750d1b49be9c3f79501a63c1ab', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7ef4b44a3f94d599eacce269b06594c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:23:26.157928 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-9d43d3a5-86e9-46d1-861b-dc5f7e55b27d tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] 10.222.0.52 "GET /compute/v2.1/servers/2ad09ab6-ef1f-47a2-bf44-6ae046cc8645" status: 200 len: 1366 microversion: 2.1 time: 0.289480 Aug 27 02:23:26.173904 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 76/152] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:23:25 2022] GET /compute/v2.1/servers/2ad09ab6-ef1f-47a2-bf44-6ae046cc8645 => generated 1366 bytes in 293 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:23:26.177269 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-5d5f743b-c7cd-438e-84e6-a240688704df tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] 10.222.0.52 "GET /compute/v2.1/servers/fbef28bb-3cdf-4a27-8493-add7806af828" status: 200 len: 1314 microversion: 2.1 time: 0.300864 Aug 27 02:23:26.195010 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 77/153] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:23:25 2022] GET /compute/v2.1/servers/fbef28bb-3cdf-4a27-8493-add7806af828 => generated 1314 bytes in 319 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:23:26.529923 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-662fd0ee-92f4-4b02-9f02-796aef9e9afb tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Calling method '>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:23:26.532248 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-662fd0ee-92f4-4b02-9f02-796aef9e9afb tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] [instance: 05bed8e9-f530-418a-9819-4f65dabdcda0] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:23:26.540642 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-662fd0ee-92f4-4b02-9f02-796aef9e9afb tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:23:26.541330 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-662fd0ee-92f4-4b02-9f02-796aef9e9afb tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:23:26.541982 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-662fd0ee-92f4-4b02-9f02-796aef9e9afb tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:23:26.542882 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-662fd0ee-92f4-4b02-9f02-796aef9e9afb tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:23:26.543553 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-662fd0ee-92f4-4b02-9f02-796aef9e9afb tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:23:26.544094 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-662fd0ee-92f4-4b02-9f02-796aef9e9afb tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:23:26.726297 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-662fd0ee-92f4-4b02-9f02-796aef9e9afb tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd945d512ae034ec08838d0a3afde2429', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '880ac4003248441c809a76cfd1b221eb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:23:26.736413 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-662fd0ee-92f4-4b02-9f02-796aef9e9afb tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] 10.222.0.52 "GET /compute/v2.1/servers/05bed8e9-f530-418a-9819-4f65dabdcda0" status: 200 len: 1322 microversion: 2.1 time: 0.210155 Aug 27 02:23:26.737228 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 77/154] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:23:26 2022] GET /compute/v2.1/servers/05bed8e9-f530-418a-9819-4f65dabdcda0 => generated 1322 bytes in 212 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 27 02:23:27.186314 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-c82ee978-96b5-40c9-8681-f1f77bc0d6b0 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:23:27.188188 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-c82ee978-96b5-40c9-8681-f1f77bc0d6b0 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] [instance: 2ad09ab6-ef1f-47a2-bf44-6ae046cc8645] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:23:27.200389 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-c82ee978-96b5-40c9-8681-f1f77bc0d6b0 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:23:27.201336 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-c82ee978-96b5-40c9-8681-f1f77bc0d6b0 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:23:27.202148 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-c82ee978-96b5-40c9-8681-f1f77bc0d6b0 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:23:27.203244 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-c82ee978-96b5-40c9-8681-f1f77bc0d6b0 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:23:27.204657 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-c82ee978-96b5-40c9-8681-f1f77bc0d6b0 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:23:27.205479 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-c82ee978-96b5-40c9-8681-f1f77bc0d6b0 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:23:27.209633 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-c7a562d6-f089-4e57-aa12-22f33344d580 tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Calling method '>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:23:27.210246 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-c7a562d6-f089-4e57-aa12-22f33344d580 tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] [instance: fbef28bb-3cdf-4a27-8493-add7806af828] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:23:27.230449 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-c7a562d6-f089-4e57-aa12-22f33344d580 tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:23:27.231273 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-c7a562d6-f089-4e57-aa12-22f33344d580 tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:23:27.232168 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-c7a562d6-f089-4e57-aa12-22f33344d580 tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:23:27.233332 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-c7a562d6-f089-4e57-aa12-22f33344d580 tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:23:27.234189 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-c7a562d6-f089-4e57-aa12-22f33344d580 tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:23:27.234991 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-c7a562d6-f089-4e57-aa12-22f33344d580 tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:23:27.472746 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-c82ee978-96b5-40c9-8681-f1f77bc0d6b0 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '26a86351583d49d090b3db09f127d49f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5d5ec8450c0941d18d63558baab54737', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:23:27.487193 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-c7a562d6-f089-4e57-aa12-22f33344d580 tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '55640f750d1b49be9c3f79501a63c1ab', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7ef4b44a3f94d599eacce269b06594c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:23:27.493041 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-c82ee978-96b5-40c9-8681-f1f77bc0d6b0 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] 10.222.0.52 "GET /compute/v2.1/servers/2ad09ab6-ef1f-47a2-bf44-6ae046cc8645" status: 200 len: 1366 microversion: 2.1 time: 0.312800 Aug 27 02:23:27.495905 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 78/155] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:23:27 2022] GET /compute/v2.1/servers/2ad09ab6-ef1f-47a2-bf44-6ae046cc8645 => generated 1366 bytes in 318 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:23:27.499137 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-c7a562d6-f089-4e57-aa12-22f33344d580 tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] 10.222.0.52 "GET /compute/v2.1/servers/fbef28bb-3cdf-4a27-8493-add7806af828" status: 200 len: 1314 microversion: 2.1 time: 0.302402 Aug 27 02:23:27.501201 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 78/156] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:23:27 2022] GET /compute/v2.1/servers/fbef28bb-3cdf-4a27-8493-add7806af828 => generated 1314 bytes in 305 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:23:27.759714 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-6c24c3ba-101e-4864-b10e-c69b52b9cb18 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:23:27.760388 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-6c24c3ba-101e-4864-b10e-c69b52b9cb18 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] [instance: 05bed8e9-f530-418a-9819-4f65dabdcda0] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:23:27.772150 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-6c24c3ba-101e-4864-b10e-c69b52b9cb18 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:23:27.772645 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-6c24c3ba-101e-4864-b10e-c69b52b9cb18 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:23:27.773096 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-6c24c3ba-101e-4864-b10e-c69b52b9cb18 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:23:27.773841 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-6c24c3ba-101e-4864-b10e-c69b52b9cb18 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:23:27.774297 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-6c24c3ba-101e-4864-b10e-c69b52b9cb18 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:23:27.774718 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-6c24c3ba-101e-4864-b10e-c69b52b9cb18 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:23:28.051033 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-6c24c3ba-101e-4864-b10e-c69b52b9cb18 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd945d512ae034ec08838d0a3afde2429', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '880ac4003248441c809a76cfd1b221eb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:23:28.068283 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-6c24c3ba-101e-4864-b10e-c69b52b9cb18 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] 10.222.0.52 "GET /compute/v2.1/servers/05bed8e9-f530-418a-9819-4f65dabdcda0" status: 200 len: 1322 microversion: 2.1 time: 0.311855 Aug 27 02:23:28.069441 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 79/157] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:23:27 2022] GET /compute/v2.1/servers/05bed8e9-f530-418a-9819-4f65dabdcda0 => generated 1322 bytes in 315 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:23:28.530969 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-3bc1e00f-8840-495c-bab7-9b0cf3c2c4c4 tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:23:28.531707 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-3bc1e00f-8840-495c-bab7-9b0cf3c2c4c4 tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] [instance: fbef28bb-3cdf-4a27-8493-add7806af828] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:23:28.532755 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-65b6aadd-ab5c-4e06-b14f-a8fdb32f3137 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Calling method '>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:23:28.533874 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-65b6aadd-ab5c-4e06-b14f-a8fdb32f3137 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] [instance: 2ad09ab6-ef1f-47a2-bf44-6ae046cc8645] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:23:28.542344 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-3bc1e00f-8840-495c-bab7-9b0cf3c2c4c4 tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:23:28.542836 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-3bc1e00f-8840-495c-bab7-9b0cf3c2c4c4 tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:23:28.543260 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-3bc1e00f-8840-495c-bab7-9b0cf3c2c4c4 tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:23:28.543957 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-3bc1e00f-8840-495c-bab7-9b0cf3c2c4c4 tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:23:28.544417 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-3bc1e00f-8840-495c-bab7-9b0cf3c2c4c4 tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:23:28.544844 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-3bc1e00f-8840-495c-bab7-9b0cf3c2c4c4 tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:23:28.553981 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG dbcounter [-] [120864] Writing DB stats nova_api:SELECT=68 {{(pid=120864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 27 02:23:28.556988 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-65b6aadd-ab5c-4e06-b14f-a8fdb32f3137 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:23:28.557628 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-65b6aadd-ab5c-4e06-b14f-a8fdb32f3137 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:23:28.558215 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-65b6aadd-ab5c-4e06-b14f-a8fdb32f3137 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:23:28.559064 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-65b6aadd-ab5c-4e06-b14f-a8fdb32f3137 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:23:28.559513 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-65b6aadd-ab5c-4e06-b14f-a8fdb32f3137 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:23:28.559929 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-65b6aadd-ab5c-4e06-b14f-a8fdb32f3137 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:23:28.861375 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-65b6aadd-ab5c-4e06-b14f-a8fdb32f3137 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '26a86351583d49d090b3db09f127d49f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5d5ec8450c0941d18d63558baab54737', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:23:28.874186 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-65b6aadd-ab5c-4e06-b14f-a8fdb32f3137 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] 10.222.0.52 "GET /compute/v2.1/servers/2ad09ab6-ef1f-47a2-bf44-6ae046cc8645" status: 200 len: 1366 microversion: 2.1 time: 0.351594 Aug 27 02:23:28.875771 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 79/158] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:23:28 2022] GET /compute/v2.1/servers/2ad09ab6-ef1f-47a2-bf44-6ae046cc8645 => generated 1366 bytes in 355 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:23:28.879072 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-3bc1e00f-8840-495c-bab7-9b0cf3c2c4c4 tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '55640f750d1b49be9c3f79501a63c1ab', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7ef4b44a3f94d599eacce269b06594c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:23:28.887708 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-3bc1e00f-8840-495c-bab7-9b0cf3c2c4c4 tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] 10.222.0.52 "GET /compute/v2.1/servers/fbef28bb-3cdf-4a27-8493-add7806af828" status: 200 len: 1314 microversion: 2.1 time: 0.360561 Aug 27 02:23:28.901739 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 80/159] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:23:28 2022] GET /compute/v2.1/servers/fbef28bb-3cdf-4a27-8493-add7806af828 => generated 1314 bytes in 367 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:23:29.095466 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-e29a6103-43a8-45c6-85d6-eb92bde1ad1c tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Calling method '>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:23:29.096654 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-e29a6103-43a8-45c6-85d6-eb92bde1ad1c tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] [instance: 05bed8e9-f530-418a-9819-4f65dabdcda0] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:23:29.121476 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-e29a6103-43a8-45c6-85d6-eb92bde1ad1c tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:23:29.122373 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-e29a6103-43a8-45c6-85d6-eb92bde1ad1c tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:23:29.123776 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-e29a6103-43a8-45c6-85d6-eb92bde1ad1c tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:23:29.127162 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-e29a6103-43a8-45c6-85d6-eb92bde1ad1c tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:23:29.129170 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-e29a6103-43a8-45c6-85d6-eb92bde1ad1c tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:23:29.129888 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-e29a6103-43a8-45c6-85d6-eb92bde1ad1c tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:23:29.410678 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-e29a6103-43a8-45c6-85d6-eb92bde1ad1c tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd945d512ae034ec08838d0a3afde2429', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '880ac4003248441c809a76cfd1b221eb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:23:29.421227 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-e29a6103-43a8-45c6-85d6-eb92bde1ad1c tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] 10.222.0.52 "GET /compute/v2.1/servers/05bed8e9-f530-418a-9819-4f65dabdcda0" status: 200 len: 1322 microversion: 2.1 time: 0.333756 Aug 27 02:23:29.422219 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 80/160] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:23:29 2022] GET /compute/v2.1/servers/05bed8e9-f530-418a-9819-4f65dabdcda0 => generated 1322 bytes in 336 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 27 02:23:29.573850 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [req-c81fe2f9-ac81-46b7-80f1-98813b034506 req-3e2b73bc-a54c-4ff6-874c-f01580e003f0 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "2ad09ab6-ef1f-47a2-bf44-6ae046cc8645", "name": "network-vif-plugged", "status": "completed", "tag": "e98d7081-9ec1-49da-9111-4786cc0b96ef"}]} {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 27 02:23:29.595570 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [req-c81fe2f9-ac81-46b7-80f1-98813b034506 req-3e2b73bc-a54c-4ff6-874c-f01580e003f0 service nova] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:23:29.596382 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [req-c81fe2f9-ac81-46b7-80f1-98813b034506 req-3e2b73bc-a54c-4ff6-874c-f01580e003f0 service nova] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:23:29.597339 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [req-c81fe2f9-ac81-46b7-80f1-98813b034506 req-3e2b73bc-a54c-4ff6-874c-f01580e003f0 service nova] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:23:29.626800 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.compute.server_external_events [req-c81fe2f9-ac81-46b7-80f1-98813b034506 req-3e2b73bc-a54c-4ff6-874c-f01580e003f0 service nova] Creating event network-vif-plugged:e98d7081-9ec1-49da-9111-4786cc0b96ef for instance 2ad09ab6-ef1f-47a2-bf44-6ae046cc8645 on hv-cinder-84630 Aug 27 02:23:29.632835 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [req-c81fe2f9-ac81-46b7-80f1-98813b034506 req-3e2b73bc-a54c-4ff6-874c-f01580e003f0 service nova] 10.222.0.52 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.470547 Aug 27 02:23:29.633486 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 81/161] 10.222.0.52 () {64 vars in 1317 bytes} [Sat Aug 27 02:23:29 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 473 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Aug 27 02:23:29.898084 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-decc29b4-feb0-4e1e-a678-d9c6f6c48569 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Calling method '>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:23:29.898683 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-decc29b4-feb0-4e1e-a678-d9c6f6c48569 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] [instance: 2ad09ab6-ef1f-47a2-bf44-6ae046cc8645] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:23:29.910599 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-d11f0209-1ec3-4aa0-955a-3da63f075da4 tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:23:29.911869 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-d11f0209-1ec3-4aa0-955a-3da63f075da4 tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] [instance: fbef28bb-3cdf-4a27-8493-add7806af828] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:23:29.912099 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-decc29b4-feb0-4e1e-a678-d9c6f6c48569 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:23:29.917030 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-decc29b4-feb0-4e1e-a678-d9c6f6c48569 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.007s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:23:29.918284 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-decc29b4-feb0-4e1e-a678-d9c6f6c48569 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:23:29.920056 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-decc29b4-feb0-4e1e-a678-d9c6f6c48569 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:23:29.920235 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-d11f0209-1ec3-4aa0-955a-3da63f075da4 tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:23:29.920908 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-decc29b4-feb0-4e1e-a678-d9c6f6c48569 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:23:29.921088 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-d11f0209-1ec3-4aa0-955a-3da63f075da4 tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:23:29.921562 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-d11f0209-1ec3-4aa0-955a-3da63f075da4 tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:23:29.921734 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-decc29b4-feb0-4e1e-a678-d9c6f6c48569 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:23:29.922287 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-d11f0209-1ec3-4aa0-955a-3da63f075da4 tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:23:29.922736 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-d11f0209-1ec3-4aa0-955a-3da63f075da4 tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:23:29.923156 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-d11f0209-1ec3-4aa0-955a-3da63f075da4 tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:23:30.164549 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-d11f0209-1ec3-4aa0-955a-3da63f075da4 tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '55640f750d1b49be9c3f79501a63c1ab', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7ef4b44a3f94d599eacce269b06594c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:23:30.169107 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-decc29b4-feb0-4e1e-a678-d9c6f6c48569 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '26a86351583d49d090b3db09f127d49f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5d5ec8450c0941d18d63558baab54737', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:23:30.178010 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-d11f0209-1ec3-4aa0-955a-3da63f075da4 tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] 10.222.0.52 "GET /compute/v2.1/servers/fbef28bb-3cdf-4a27-8493-add7806af828" status: 200 len: 1314 microversion: 2.1 time: 0.270863 Aug 27 02:23:30.180335 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 82/162] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:23:29 2022] GET /compute/v2.1/servers/fbef28bb-3cdf-4a27-8493-add7806af828 => generated 1314 bytes in 275 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:23:30.190578 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-decc29b4-feb0-4e1e-a678-d9c6f6c48569 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] 10.222.0.52 "GET /compute/v2.1/servers/2ad09ab6-ef1f-47a2-bf44-6ae046cc8645" status: 200 len: 1366 microversion: 2.1 time: 0.296967 Aug 27 02:23:30.193983 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 81/163] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:23:29 2022] GET /compute/v2.1/servers/2ad09ab6-ef1f-47a2-bf44-6ae046cc8645 => generated 1366 bytes in 298 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:23:30.455609 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-afeafa3f-f425-4f36-be09-cb91ce3c6fd3 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:23:30.457318 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-afeafa3f-f425-4f36-be09-cb91ce3c6fd3 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] [instance: 05bed8e9-f530-418a-9819-4f65dabdcda0] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:23:30.466345 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-afeafa3f-f425-4f36-be09-cb91ce3c6fd3 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:23:30.467331 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-afeafa3f-f425-4f36-be09-cb91ce3c6fd3 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:23:30.468131 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-afeafa3f-f425-4f36-be09-cb91ce3c6fd3 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:23:30.469381 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-afeafa3f-f425-4f36-be09-cb91ce3c6fd3 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:23:30.470254 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-afeafa3f-f425-4f36-be09-cb91ce3c6fd3 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:23:30.471073 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-afeafa3f-f425-4f36-be09-cb91ce3c6fd3 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:23:30.940511 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-afeafa3f-f425-4f36-be09-cb91ce3c6fd3 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd945d512ae034ec08838d0a3afde2429', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '880ac4003248441c809a76cfd1b221eb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:23:30.976223 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-afeafa3f-f425-4f36-be09-cb91ce3c6fd3 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] 10.222.0.52 "GET /compute/v2.1/servers/05bed8e9-f530-418a-9819-4f65dabdcda0" status: 200 len: 1322 microversion: 2.1 time: 0.524583 Aug 27 02:23:30.976808 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 83/164] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:23:30 2022] GET /compute/v2.1/servers/05bed8e9-f530-418a-9819-4f65dabdcda0 => generated 1322 bytes in 526 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 27 02:23:31.202477 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-582863f6-34d0-460f-81a3-e7fef0b8e134 tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Calling method '>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:23:31.203247 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-582863f6-34d0-460f-81a3-e7fef0b8e134 tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] [instance: fbef28bb-3cdf-4a27-8493-add7806af828] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:23:31.215922 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-582863f6-34d0-460f-81a3-e7fef0b8e134 tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:23:31.218018 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-582863f6-34d0-460f-81a3-e7fef0b8e134 tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:23:31.218978 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-582863f6-34d0-460f-81a3-e7fef0b8e134 tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:23:31.220501 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-970af0ba-d768-43e6-be51-a26b068491e4 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:23:31.221790 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-970af0ba-d768-43e6-be51-a26b068491e4 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] [instance: 2ad09ab6-ef1f-47a2-bf44-6ae046cc8645] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:23:31.222043 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-582863f6-34d0-460f-81a3-e7fef0b8e134 tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:23:31.222623 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-582863f6-34d0-460f-81a3-e7fef0b8e134 tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:23:31.223091 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-582863f6-34d0-460f-81a3-e7fef0b8e134 tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:23:31.247831 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-970af0ba-d768-43e6-be51-a26b068491e4 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:23:31.250337 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-970af0ba-d768-43e6-be51-a26b068491e4 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:23:31.253197 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-970af0ba-d768-43e6-be51-a26b068491e4 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:23:31.260598 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-970af0ba-d768-43e6-be51-a26b068491e4 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:23:31.261603 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-970af0ba-d768-43e6-be51-a26b068491e4 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:23:31.262486 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-970af0ba-d768-43e6-be51-a26b068491e4 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:23:31.847880 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-582863f6-34d0-460f-81a3-e7fef0b8e134 tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '55640f750d1b49be9c3f79501a63c1ab', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7ef4b44a3f94d599eacce269b06594c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:23:31.870203 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-582863f6-34d0-460f-81a3-e7fef0b8e134 tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] 10.222.0.52 "GET /compute/v2.1/servers/fbef28bb-3cdf-4a27-8493-add7806af828" status: 200 len: 1314 microversion: 2.1 time: 0.671711 Aug 27 02:23:31.877396 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 82/165] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:23:31 2022] GET /compute/v2.1/servers/fbef28bb-3cdf-4a27-8493-add7806af828 => generated 1314 bytes in 680 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:23:32.014739 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-e54f7c6c-507d-4845-9b25-b3ce8afa7186 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Calling method '>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:23:32.016224 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-e54f7c6c-507d-4845-9b25-b3ce8afa7186 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] [instance: 05bed8e9-f530-418a-9819-4f65dabdcda0] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:23:32.026475 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-e54f7c6c-507d-4845-9b25-b3ce8afa7186 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:23:32.026964 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-e54f7c6c-507d-4845-9b25-b3ce8afa7186 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:23:32.027555 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-e54f7c6c-507d-4845-9b25-b3ce8afa7186 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:23:32.032896 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-e54f7c6c-507d-4845-9b25-b3ce8afa7186 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:23:32.033976 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-e54f7c6c-507d-4845-9b25-b3ce8afa7186 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:23:32.034657 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-e54f7c6c-507d-4845-9b25-b3ce8afa7186 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:23:32.165799 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-970af0ba-d768-43e6-be51-a26b068491e4 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '26a86351583d49d090b3db09f127d49f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5d5ec8450c0941d18d63558baab54737', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:23:32.174949 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-970af0ba-d768-43e6-be51-a26b068491e4 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] 10.222.0.52 "GET /compute/v2.1/servers/2ad09ab6-ef1f-47a2-bf44-6ae046cc8645" status: 200 len: 1366 microversion: 2.1 time: 0.962272 Aug 27 02:23:32.189720 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 84/166] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:23:31 2022] GET /compute/v2.1/servers/2ad09ab6-ef1f-47a2-bf44-6ae046cc8645 => generated 1366 bytes in 969 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:23:32.653552 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-e54f7c6c-507d-4845-9b25-b3ce8afa7186 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd945d512ae034ec08838d0a3afde2429', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '880ac4003248441c809a76cfd1b221eb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:23:32.691810 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-e54f7c6c-507d-4845-9b25-b3ce8afa7186 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] 10.222.0.52 "GET /compute/v2.1/servers/05bed8e9-f530-418a-9819-4f65dabdcda0" status: 200 len: 1322 microversion: 2.1 time: 0.680691 Aug 27 02:23:32.697251 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 83/167] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:23:32 2022] GET /compute/v2.1/servers/05bed8e9-f530-418a-9819-4f65dabdcda0 => generated 1322 bytes in 687 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 27 02:23:32.900419 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-cb9e9f9f-c81d-42f0-97c4-138b6ae134c2 tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:23:32.901308 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-cb9e9f9f-c81d-42f0-97c4-138b6ae134c2 tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] [instance: fbef28bb-3cdf-4a27-8493-add7806af828] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:23:32.910836 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-cb9e9f9f-c81d-42f0-97c4-138b6ae134c2 tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:23:32.915443 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-cb9e9f9f-c81d-42f0-97c4-138b6ae134c2 tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:23:32.916144 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-cb9e9f9f-c81d-42f0-97c4-138b6ae134c2 tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:23:32.917672 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-cb9e9f9f-c81d-42f0-97c4-138b6ae134c2 tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:23:32.918368 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-cb9e9f9f-c81d-42f0-97c4-138b6ae134c2 tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:23:32.921641 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-cb9e9f9f-c81d-42f0-97c4-138b6ae134c2 tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:23:32.935094 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG dbcounter [-] [120863] Writing DB stats nova_cell1:SELECT=100 {{(pid=120863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 27 02:23:33.183087 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-cb9e9f9f-c81d-42f0-97c4-138b6ae134c2 tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '55640f750d1b49be9c3f79501a63c1ab', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7ef4b44a3f94d599eacce269b06594c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:23:33.191405 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-cb9e9f9f-c81d-42f0-97c4-138b6ae134c2 tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] 10.222.0.52 "GET /compute/v2.1/servers/fbef28bb-3cdf-4a27-8493-add7806af828" status: 200 len: 1314 microversion: 2.1 time: 0.295381 Aug 27 02:23:33.199917 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 85/168] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:23:32 2022] GET /compute/v2.1/servers/fbef28bb-3cdf-4a27-8493-add7806af828 => generated 1314 bytes in 304 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 27 02:23:33.213364 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-67fa6d05-bcdd-453a-8bf6-1cd35b495eb3 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Calling method '>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:23:33.213977 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-67fa6d05-bcdd-453a-8bf6-1cd35b495eb3 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] [instance: 2ad09ab6-ef1f-47a2-bf44-6ae046cc8645] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:23:33.222211 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-67fa6d05-bcdd-453a-8bf6-1cd35b495eb3 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:23:33.223547 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-67fa6d05-bcdd-453a-8bf6-1cd35b495eb3 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:23:33.224966 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-67fa6d05-bcdd-453a-8bf6-1cd35b495eb3 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:23:33.226940 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-67fa6d05-bcdd-453a-8bf6-1cd35b495eb3 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:23:33.227795 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-67fa6d05-bcdd-453a-8bf6-1cd35b495eb3 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:23:33.228975 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-67fa6d05-bcdd-453a-8bf6-1cd35b495eb3 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:23:33.364205 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [req-776163b1-5461-4f01-b046-ed8373b46cbc req-0efe9fab-8f79-404e-8bc7-1ae7b0fdd479 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "fbef28bb-3cdf-4a27-8493-add7806af828", "name": "network-vif-plugged", "status": "completed", "tag": "b4168fa6-3b0a-4107-83cf-7d31f29910d4"}]} {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 27 02:23:33.396068 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [req-776163b1-5461-4f01-b046-ed8373b46cbc req-0efe9fab-8f79-404e-8bc7-1ae7b0fdd479 service nova] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:23:33.396559 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [req-776163b1-5461-4f01-b046-ed8373b46cbc req-0efe9fab-8f79-404e-8bc7-1ae7b0fdd479 service nova] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:23:33.397008 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [req-776163b1-5461-4f01-b046-ed8373b46cbc req-0efe9fab-8f79-404e-8bc7-1ae7b0fdd479 service nova] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:23:33.425154 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.compute.server_external_events [req-776163b1-5461-4f01-b046-ed8373b46cbc req-0efe9fab-8f79-404e-8bc7-1ae7b0fdd479 service nova] Creating event network-vif-plugged:b4168fa6-3b0a-4107-83cf-7d31f29910d4 for instance fbef28bb-3cdf-4a27-8493-add7806af828 on hv-cinder-84630 Aug 27 02:23:33.433583 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [req-776163b1-5461-4f01-b046-ed8373b46cbc req-0efe9fab-8f79-404e-8bc7-1ae7b0fdd479 service nova] 10.222.0.52 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.079655 Aug 27 02:23:33.435428 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 86/169] 10.222.0.52 () {64 vars in 1317 bytes} [Sat Aug 27 02:23:33 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 82 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 27 02:23:33.656277 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-67fa6d05-bcdd-453a-8bf6-1cd35b495eb3 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '26a86351583d49d090b3db09f127d49f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5d5ec8450c0941d18d63558baab54737', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:23:33.675651 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-67fa6d05-bcdd-453a-8bf6-1cd35b495eb3 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] 10.222.0.52 "GET /compute/v2.1/servers/2ad09ab6-ef1f-47a2-bf44-6ae046cc8645" status: 200 len: 1366 microversion: 2.1 time: 0.477241 Aug 27 02:23:33.685183 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 84/170] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:23:33 2022] GET /compute/v2.1/servers/2ad09ab6-ef1f-47a2-bf44-6ae046cc8645 => generated 1366 bytes in 479 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:23:33.719480 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-ae16261b-9a68-4fe2-8434-03c2dc2e84da tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:23:33.720337 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-ae16261b-9a68-4fe2-8434-03c2dc2e84da tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] [instance: 05bed8e9-f530-418a-9819-4f65dabdcda0] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:23:33.728206 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-ae16261b-9a68-4fe2-8434-03c2dc2e84da tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:23:33.728583 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-ae16261b-9a68-4fe2-8434-03c2dc2e84da tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:23:33.729001 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-ae16261b-9a68-4fe2-8434-03c2dc2e84da tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:23:33.729723 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-ae16261b-9a68-4fe2-8434-03c2dc2e84da tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:23:33.730167 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-ae16261b-9a68-4fe2-8434-03c2dc2e84da tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:23:33.730726 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-ae16261b-9a68-4fe2-8434-03c2dc2e84da tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:23:33.922865 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-ae16261b-9a68-4fe2-8434-03c2dc2e84da tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd945d512ae034ec08838d0a3afde2429', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '880ac4003248441c809a76cfd1b221eb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:23:33.932643 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-ae16261b-9a68-4fe2-8434-03c2dc2e84da tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] 10.222.0.52 "GET /compute/v2.1/servers/05bed8e9-f530-418a-9819-4f65dabdcda0" status: 200 len: 1322 microversion: 2.1 time: 0.217258 Aug 27 02:23:33.937157 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 87/171] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:23:33 2022] GET /compute/v2.1/servers/05bed8e9-f530-418a-9819-4f65dabdcda0 => generated 1322 bytes in 220 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:23:34.233731 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-ced72a2e-8ee0-4503-b3cf-8ab93af1d855 tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Calling method '>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:23:34.234420 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-ced72a2e-8ee0-4503-b3cf-8ab93af1d855 tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] [instance: fbef28bb-3cdf-4a27-8493-add7806af828] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:23:34.245348 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-ced72a2e-8ee0-4503-b3cf-8ab93af1d855 tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:23:34.247924 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-ced72a2e-8ee0-4503-b3cf-8ab93af1d855 tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:23:34.249749 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-ced72a2e-8ee0-4503-b3cf-8ab93af1d855 tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:23:34.252949 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-ced72a2e-8ee0-4503-b3cf-8ab93af1d855 tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:23:34.253957 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-ced72a2e-8ee0-4503-b3cf-8ab93af1d855 tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:23:34.255574 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-ced72a2e-8ee0-4503-b3cf-8ab93af1d855 tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:23:34.507326 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-ced72a2e-8ee0-4503-b3cf-8ab93af1d855 tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '55640f750d1b49be9c3f79501a63c1ab', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7ef4b44a3f94d599eacce269b06594c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:23:34.519425 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-ced72a2e-8ee0-4503-b3cf-8ab93af1d855 tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] 10.222.0.52 "GET /compute/v2.1/servers/fbef28bb-3cdf-4a27-8493-add7806af828" status: 200 len: 1314 microversion: 2.1 time: 0.289769 Aug 27 02:23:34.520319 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 85/172] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:23:34 2022] GET /compute/v2.1/servers/fbef28bb-3cdf-4a27-8493-add7806af828 => generated 1314 bytes in 292 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:23:34.716336 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-2f7156ec-8fa3-4158-836a-9c5b1eed79b1 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:23:34.717011 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-2f7156ec-8fa3-4158-836a-9c5b1eed79b1 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] [instance: 2ad09ab6-ef1f-47a2-bf44-6ae046cc8645] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:23:34.725289 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-2f7156ec-8fa3-4158-836a-9c5b1eed79b1 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:23:34.727028 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-2f7156ec-8fa3-4158-836a-9c5b1eed79b1 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:23:34.727635 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-2f7156ec-8fa3-4158-836a-9c5b1eed79b1 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:23:34.729523 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-2f7156ec-8fa3-4158-836a-9c5b1eed79b1 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:23:34.730261 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-2f7156ec-8fa3-4158-836a-9c5b1eed79b1 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:23:34.730997 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-2f7156ec-8fa3-4158-836a-9c5b1eed79b1 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:23:34.943890 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-2f7156ec-8fa3-4158-836a-9c5b1eed79b1 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '26a86351583d49d090b3db09f127d49f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5d5ec8450c0941d18d63558baab54737', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:23:34.959952 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-2f7156ec-8fa3-4158-836a-9c5b1eed79b1 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] 10.222.0.52 "GET /compute/v2.1/servers/2ad09ab6-ef1f-47a2-bf44-6ae046cc8645" status: 200 len: 1366 microversion: 2.1 time: 0.253338 Aug 27 02:23:34.961618 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 88/173] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:23:34 2022] GET /compute/v2.1/servers/2ad09ab6-ef1f-47a2-bf44-6ae046cc8645 => generated 1366 bytes in 256 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:23:34.977815 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-fb316c3b-71f1-478a-b4a1-e7e66d9d5a43 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Calling method '>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:23:34.978849 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-fb316c3b-71f1-478a-b4a1-e7e66d9d5a43 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] [instance: 05bed8e9-f530-418a-9819-4f65dabdcda0] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:23:34.987445 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-fb316c3b-71f1-478a-b4a1-e7e66d9d5a43 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:23:34.988329 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-fb316c3b-71f1-478a-b4a1-e7e66d9d5a43 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:23:34.989056 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-fb316c3b-71f1-478a-b4a1-e7e66d9d5a43 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:23:34.996408 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-fb316c3b-71f1-478a-b4a1-e7e66d9d5a43 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:23:34.997330 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-fb316c3b-71f1-478a-b4a1-e7e66d9d5a43 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.007s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:23:34.997916 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-fb316c3b-71f1-478a-b4a1-e7e66d9d5a43 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:23:35.186986 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-fb316c3b-71f1-478a-b4a1-e7e66d9d5a43 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd945d512ae034ec08838d0a3afde2429', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '880ac4003248441c809a76cfd1b221eb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:23:35.195195 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-fb316c3b-71f1-478a-b4a1-e7e66d9d5a43 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] 10.222.0.52 "GET /compute/v2.1/servers/05bed8e9-f530-418a-9819-4f65dabdcda0" status: 200 len: 1322 microversion: 2.1 time: 0.235033 Aug 27 02:23:35.209319 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 86/174] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:23:34 2022] GET /compute/v2.1/servers/05bed8e9-f530-418a-9819-4f65dabdcda0 => generated 1322 bytes in 250 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:23:35.545496 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-0115e206-6abc-4841-9cd3-331c366446cb tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:23:35.546416 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-0115e206-6abc-4841-9cd3-331c366446cb tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] [instance: fbef28bb-3cdf-4a27-8493-add7806af828] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:23:35.559060 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-0115e206-6abc-4841-9cd3-331c366446cb tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:23:35.559549 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-0115e206-6abc-4841-9cd3-331c366446cb tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:23:35.560779 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-0115e206-6abc-4841-9cd3-331c366446cb tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:23:35.561443 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-0115e206-6abc-4841-9cd3-331c366446cb tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:23:35.564492 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-0115e206-6abc-4841-9cd3-331c366446cb tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:23:35.565811 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-0115e206-6abc-4841-9cd3-331c366446cb tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:23:35.840819 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-0115e206-6abc-4841-9cd3-331c366446cb tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '55640f750d1b49be9c3f79501a63c1ab', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7ef4b44a3f94d599eacce269b06594c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:23:35.851153 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-0115e206-6abc-4841-9cd3-331c366446cb tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] 10.222.0.52 "GET /compute/v2.1/servers/fbef28bb-3cdf-4a27-8493-add7806af828" status: 200 len: 1314 microversion: 2.1 time: 0.310580 Aug 27 02:23:35.852008 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 89/175] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:23:35 2022] GET /compute/v2.1/servers/fbef28bb-3cdf-4a27-8493-add7806af828 => generated 1314 bytes in 313 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:23:36.010313 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-665bbfc7-0d04-44d1-a208-94a4ce079fc3 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Calling method '>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:23:36.013015 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-665bbfc7-0d04-44d1-a208-94a4ce079fc3 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] [instance: 2ad09ab6-ef1f-47a2-bf44-6ae046cc8645] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:23:36.031150 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-665bbfc7-0d04-44d1-a208-94a4ce079fc3 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:23:36.033601 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-665bbfc7-0d04-44d1-a208-94a4ce079fc3 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:23:36.035445 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-665bbfc7-0d04-44d1-a208-94a4ce079fc3 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:23:36.036727 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-665bbfc7-0d04-44d1-a208-94a4ce079fc3 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:23:36.037940 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-665bbfc7-0d04-44d1-a208-94a4ce079fc3 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:23:36.038731 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-665bbfc7-0d04-44d1-a208-94a4ce079fc3 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:23:36.224143 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-40be4b4a-25f1-4be7-b00b-c35dd3610c1a tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:23:36.233830 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-40be4b4a-25f1-4be7-b00b-c35dd3610c1a tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] [instance: 05bed8e9-f530-418a-9819-4f65dabdcda0] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:23:36.249674 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-40be4b4a-25f1-4be7-b00b-c35dd3610c1a tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:23:36.250456 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-40be4b4a-25f1-4be7-b00b-c35dd3610c1a tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:23:36.251141 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-40be4b4a-25f1-4be7-b00b-c35dd3610c1a tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:23:36.252281 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-40be4b4a-25f1-4be7-b00b-c35dd3610c1a tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:23:36.253019 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-40be4b4a-25f1-4be7-b00b-c35dd3610c1a tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:23:36.261396 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-40be4b4a-25f1-4be7-b00b-c35dd3610c1a tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:23:36.389700 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-665bbfc7-0d04-44d1-a208-94a4ce079fc3 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '26a86351583d49d090b3db09f127d49f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5d5ec8450c0941d18d63558baab54737', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:23:36.414236 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-665bbfc7-0d04-44d1-a208-94a4ce079fc3 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] 10.222.0.52 "GET /compute/v2.1/servers/2ad09ab6-ef1f-47a2-bf44-6ae046cc8645" status: 200 len: 1366 microversion: 2.1 time: 0.431664 Aug 27 02:23:36.421910 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 87/176] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:23:35 2022] GET /compute/v2.1/servers/2ad09ab6-ef1f-47a2-bf44-6ae046cc8645 => generated 1366 bytes in 441 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:23:36.648995 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-40be4b4a-25f1-4be7-b00b-c35dd3610c1a tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd945d512ae034ec08838d0a3afde2429', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '880ac4003248441c809a76cfd1b221eb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:23:36.662993 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-40be4b4a-25f1-4be7-b00b-c35dd3610c1a tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] 10.222.0.52 "GET /compute/v2.1/servers/05bed8e9-f530-418a-9819-4f65dabdcda0" status: 200 len: 1322 microversion: 2.1 time: 0.448193 Aug 27 02:23:36.664363 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 90/177] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:23:36 2022] GET /compute/v2.1/servers/05bed8e9-f530-418a-9819-4f65dabdcda0 => generated 1322 bytes in 451 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 27 02:23:36.873882 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-546f1899-2ad6-4757-953c-ecc0d9d0c861 tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Calling method '>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:23:36.875097 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-546f1899-2ad6-4757-953c-ecc0d9d0c861 tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] [instance: fbef28bb-3cdf-4a27-8493-add7806af828] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:23:36.882296 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-546f1899-2ad6-4757-953c-ecc0d9d0c861 tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:23:36.882782 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-546f1899-2ad6-4757-953c-ecc0d9d0c861 tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:23:36.883213 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-546f1899-2ad6-4757-953c-ecc0d9d0c861 tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:23:36.883959 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-546f1899-2ad6-4757-953c-ecc0d9d0c861 tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:23:36.884386 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-546f1899-2ad6-4757-953c-ecc0d9d0c861 tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:23:36.884972 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-546f1899-2ad6-4757-953c-ecc0d9d0c861 tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:23:37.061964 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-546f1899-2ad6-4757-953c-ecc0d9d0c861 tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '55640f750d1b49be9c3f79501a63c1ab', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7ef4b44a3f94d599eacce269b06594c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:23:37.071107 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-546f1899-2ad6-4757-953c-ecc0d9d0c861 tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] 10.222.0.52 "GET /compute/v2.1/servers/fbef28bb-3cdf-4a27-8493-add7806af828" status: 200 len: 1314 microversion: 2.1 time: 0.200914 Aug 27 02:23:37.071686 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 88/178] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:23:36 2022] GET /compute/v2.1/servers/fbef28bb-3cdf-4a27-8493-add7806af828 => generated 1314 bytes in 202 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:23:37.444594 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-f6101b33-77cd-4867-a77c-eccddd9b37e8 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:23:37.445794 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-f6101b33-77cd-4867-a77c-eccddd9b37e8 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] [instance: 2ad09ab6-ef1f-47a2-bf44-6ae046cc8645] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:23:37.452719 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-f6101b33-77cd-4867-a77c-eccddd9b37e8 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:23:37.453390 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-f6101b33-77cd-4867-a77c-eccddd9b37e8 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:23:37.453826 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-f6101b33-77cd-4867-a77c-eccddd9b37e8 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:23:37.454532 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-f6101b33-77cd-4867-a77c-eccddd9b37e8 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:23:37.454984 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-f6101b33-77cd-4867-a77c-eccddd9b37e8 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:23:37.455429 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-f6101b33-77cd-4867-a77c-eccddd9b37e8 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:23:37.693276 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-504303fa-f67e-4133-b4b3-1c5516d47f0a tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Calling method '>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:23:37.694636 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-504303fa-f67e-4133-b4b3-1c5516d47f0a tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] [instance: 05bed8e9-f530-418a-9819-4f65dabdcda0] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:23:37.708438 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-f6101b33-77cd-4867-a77c-eccddd9b37e8 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '26a86351583d49d090b3db09f127d49f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5d5ec8450c0941d18d63558baab54737', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:23:37.717605 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-f6101b33-77cd-4867-a77c-eccddd9b37e8 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] 10.222.0.52 "GET /compute/v2.1/servers/2ad09ab6-ef1f-47a2-bf44-6ae046cc8645" status: 200 len: 1366 microversion: 2.1 time: 0.277835 Aug 27 02:23:37.720988 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 91/179] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:23:37 2022] GET /compute/v2.1/servers/2ad09ab6-ef1f-47a2-bf44-6ae046cc8645 => generated 1366 bytes in 282 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:23:37.732194 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-504303fa-f67e-4133-b4b3-1c5516d47f0a tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:23:37.733075 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-504303fa-f67e-4133-b4b3-1c5516d47f0a tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:23:37.733872 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-504303fa-f67e-4133-b4b3-1c5516d47f0a tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:23:37.735113 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-504303fa-f67e-4133-b4b3-1c5516d47f0a tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:23:37.735930 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-504303fa-f67e-4133-b4b3-1c5516d47f0a tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:23:37.736692 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-504303fa-f67e-4133-b4b3-1c5516d47f0a tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:23:38.094899 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-02d134e2-f259-4f23-943f-598aca7c179d tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:23:38.095476 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-02d134e2-f259-4f23-943f-598aca7c179d tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] [instance: fbef28bb-3cdf-4a27-8493-add7806af828] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:23:38.107825 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-02d134e2-f259-4f23-943f-598aca7c179d tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:23:38.112888 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-02d134e2-f259-4f23-943f-598aca7c179d tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:23:38.113691 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-02d134e2-f259-4f23-943f-598aca7c179d tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:23:38.114877 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-02d134e2-f259-4f23-943f-598aca7c179d tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:23:38.115504 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-02d134e2-f259-4f23-943f-598aca7c179d tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:23:38.116112 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-02d134e2-f259-4f23-943f-598aca7c179d tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:23:38.177525 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-504303fa-f67e-4133-b4b3-1c5516d47f0a tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd945d512ae034ec08838d0a3afde2429', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '880ac4003248441c809a76cfd1b221eb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:23:38.204116 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-504303fa-f67e-4133-b4b3-1c5516d47f0a tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] 10.222.0.52 "GET /compute/v2.1/servers/05bed8e9-f530-418a-9819-4f65dabdcda0" status: 200 len: 1322 microversion: 2.1 time: 0.519642 Aug 27 02:23:38.205552 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 89/180] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:23:37 2022] GET /compute/v2.1/servers/05bed8e9-f530-418a-9819-4f65dabdcda0 => generated 1322 bytes in 522 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:23:38.384090 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-02d134e2-f259-4f23-943f-598aca7c179d tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '55640f750d1b49be9c3f79501a63c1ab', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7ef4b44a3f94d599eacce269b06594c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:23:38.394170 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-02d134e2-f259-4f23-943f-598aca7c179d tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] 10.222.0.52 "GET /compute/v2.1/servers/fbef28bb-3cdf-4a27-8493-add7806af828" status: 200 len: 1314 microversion: 2.1 time: 0.303135 Aug 27 02:23:38.395010 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 92/181] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:23:38 2022] GET /compute/v2.1/servers/fbef28bb-3cdf-4a27-8493-add7806af828 => generated 1314 bytes in 306 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 27 02:23:38.763301 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-49f3ab37-48f5-4f23-9725-514d8756e213 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Calling method '>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:23:38.763903 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-49f3ab37-48f5-4f23-9725-514d8756e213 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] [instance: 2ad09ab6-ef1f-47a2-bf44-6ae046cc8645] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:23:38.775750 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-49f3ab37-48f5-4f23-9725-514d8756e213 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:23:38.776918 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-49f3ab37-48f5-4f23-9725-514d8756e213 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:23:38.777925 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-49f3ab37-48f5-4f23-9725-514d8756e213 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:23:38.779148 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-49f3ab37-48f5-4f23-9725-514d8756e213 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:23:38.779950 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-49f3ab37-48f5-4f23-9725-514d8756e213 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:23:38.780681 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-49f3ab37-48f5-4f23-9725-514d8756e213 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:23:38.992252 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-49f3ab37-48f5-4f23-9725-514d8756e213 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '26a86351583d49d090b3db09f127d49f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5d5ec8450c0941d18d63558baab54737', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:23:39.003784 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-49f3ab37-48f5-4f23-9725-514d8756e213 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] 10.222.0.52 "GET /compute/v2.1/servers/2ad09ab6-ef1f-47a2-bf44-6ae046cc8645" status: 200 len: 1366 microversion: 2.1 time: 0.248187 Aug 27 02:23:39.014442 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 90/182] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:23:38 2022] GET /compute/v2.1/servers/2ad09ab6-ef1f-47a2-bf44-6ae046cc8645 => generated 1366 bytes in 259 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 27 02:23:39.252859 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-b4cb119f-0581-42c0-a8be-e965c1e26362 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:23:39.256366 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-b4cb119f-0581-42c0-a8be-e965c1e26362 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] [instance: 05bed8e9-f530-418a-9819-4f65dabdcda0] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:23:39.274857 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-b4cb119f-0581-42c0-a8be-e965c1e26362 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:23:39.275518 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-b4cb119f-0581-42c0-a8be-e965c1e26362 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:23:39.276100 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-b4cb119f-0581-42c0-a8be-e965c1e26362 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:23:39.277251 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-b4cb119f-0581-42c0-a8be-e965c1e26362 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:23:39.277847 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-b4cb119f-0581-42c0-a8be-e965c1e26362 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:23:39.278410 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-b4cb119f-0581-42c0-a8be-e965c1e26362 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:23:39.418009 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-40673388-3407-4e9c-ae61-0b727294ba31 tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Calling method '>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:23:39.418739 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-40673388-3407-4e9c-ae61-0b727294ba31 tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] [instance: fbef28bb-3cdf-4a27-8493-add7806af828] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:23:39.430232 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-40673388-3407-4e9c-ae61-0b727294ba31 tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:23:39.430973 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-40673388-3407-4e9c-ae61-0b727294ba31 tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:23:39.431606 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-40673388-3407-4e9c-ae61-0b727294ba31 tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:23:39.432532 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-40673388-3407-4e9c-ae61-0b727294ba31 tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:23:39.433304 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-40673388-3407-4e9c-ae61-0b727294ba31 tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:23:39.433925 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-40673388-3407-4e9c-ae61-0b727294ba31 tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:23:39.515370 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-b4cb119f-0581-42c0-a8be-e965c1e26362 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd945d512ae034ec08838d0a3afde2429', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '880ac4003248441c809a76cfd1b221eb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:23:39.527487 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-b4cb119f-0581-42c0-a8be-e965c1e26362 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] 10.222.0.52 "GET /compute/v2.1/servers/05bed8e9-f530-418a-9819-4f65dabdcda0" status: 200 len: 1322 microversion: 2.1 time: 0.282387 Aug 27 02:23:39.528882 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 93/183] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:23:39 2022] GET /compute/v2.1/servers/05bed8e9-f530-418a-9819-4f65dabdcda0 => generated 1322 bytes in 285 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:23:39.667834 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-40673388-3407-4e9c-ae61-0b727294ba31 tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '55640f750d1b49be9c3f79501a63c1ab', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7ef4b44a3f94d599eacce269b06594c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:23:39.680703 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-40673388-3407-4e9c-ae61-0b727294ba31 tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] 10.222.0.52 "GET /compute/v2.1/servers/fbef28bb-3cdf-4a27-8493-add7806af828" status: 200 len: 1314 microversion: 2.1 time: 0.267220 Aug 27 02:23:39.681870 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 91/184] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:23:39 2022] GET /compute/v2.1/servers/fbef28bb-3cdf-4a27-8493-add7806af828 => generated 1314 bytes in 269 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 27 02:23:40.039115 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-213b5d89-d389-4487-b21d-541a5dbeb3e8 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:23:40.040603 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-213b5d89-d389-4487-b21d-541a5dbeb3e8 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] [instance: 2ad09ab6-ef1f-47a2-bf44-6ae046cc8645] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:23:40.049939 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-213b5d89-d389-4487-b21d-541a5dbeb3e8 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:23:40.050714 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-213b5d89-d389-4487-b21d-541a5dbeb3e8 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:23:40.051397 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-213b5d89-d389-4487-b21d-541a5dbeb3e8 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:23:40.052906 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-213b5d89-d389-4487-b21d-541a5dbeb3e8 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:23:40.053493 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-213b5d89-d389-4487-b21d-541a5dbeb3e8 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:23:40.054062 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-213b5d89-d389-4487-b21d-541a5dbeb3e8 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:23:40.348576 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-213b5d89-d389-4487-b21d-541a5dbeb3e8 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '26a86351583d49d090b3db09f127d49f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5d5ec8450c0941d18d63558baab54737', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:23:40.361599 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-213b5d89-d389-4487-b21d-541a5dbeb3e8 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] 10.222.0.52 "GET /compute/v2.1/servers/2ad09ab6-ef1f-47a2-bf44-6ae046cc8645" status: 200 len: 1366 microversion: 2.1 time: 0.336009 Aug 27 02:23:40.366984 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 94/185] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:23:40 2022] GET /compute/v2.1/servers/2ad09ab6-ef1f-47a2-bf44-6ae046cc8645 => generated 1366 bytes in 342 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:23:40.549923 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-f04d324d-02af-408f-bff8-9e79b6d04f8f tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Calling method '>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:23:40.550696 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-f04d324d-02af-408f-bff8-9e79b6d04f8f tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] [instance: 05bed8e9-f530-418a-9819-4f65dabdcda0] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:23:40.563500 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-f04d324d-02af-408f-bff8-9e79b6d04f8f tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:23:40.565101 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-f04d324d-02af-408f-bff8-9e79b6d04f8f tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:23:40.567587 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-f04d324d-02af-408f-bff8-9e79b6d04f8f tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:23:40.568461 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-f04d324d-02af-408f-bff8-9e79b6d04f8f tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:23:40.569002 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-f04d324d-02af-408f-bff8-9e79b6d04f8f tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:23:40.572587 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-f04d324d-02af-408f-bff8-9e79b6d04f8f tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:23:40.713814 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-22ec9f9c-2f46-4cc6-91cf-34797b665b6b tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:23:40.717868 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-22ec9f9c-2f46-4cc6-91cf-34797b665b6b tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] [instance: fbef28bb-3cdf-4a27-8493-add7806af828] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:23:40.730085 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-22ec9f9c-2f46-4cc6-91cf-34797b665b6b tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:23:40.730966 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-22ec9f9c-2f46-4cc6-91cf-34797b665b6b tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:23:40.734227 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-22ec9f9c-2f46-4cc6-91cf-34797b665b6b tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:23:40.735623 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-22ec9f9c-2f46-4cc6-91cf-34797b665b6b tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:23:40.736417 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-22ec9f9c-2f46-4cc6-91cf-34797b665b6b tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:23:40.737897 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-22ec9f9c-2f46-4cc6-91cf-34797b665b6b tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:23:40.863599 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-f04d324d-02af-408f-bff8-9e79b6d04f8f tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd945d512ae034ec08838d0a3afde2429', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '880ac4003248441c809a76cfd1b221eb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:23:40.873138 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-f04d324d-02af-408f-bff8-9e79b6d04f8f tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] 10.222.0.52 "GET /compute/v2.1/servers/05bed8e9-f530-418a-9819-4f65dabdcda0" status: 200 len: 1322 microversion: 2.1 time: 0.327656 Aug 27 02:23:40.875062 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 92/186] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:23:40 2022] GET /compute/v2.1/servers/05bed8e9-f530-418a-9819-4f65dabdcda0 => generated 1322 bytes in 330 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:23:40.935909 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-22ec9f9c-2f46-4cc6-91cf-34797b665b6b tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '55640f750d1b49be9c3f79501a63c1ab', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7ef4b44a3f94d599eacce269b06594c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:23:40.948274 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-22ec9f9c-2f46-4cc6-91cf-34797b665b6b tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] 10.222.0.52 "GET /compute/v2.1/servers/fbef28bb-3cdf-4a27-8493-add7806af828" status: 200 len: 1314 microversion: 2.1 time: 0.242673 Aug 27 02:23:40.962021 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 95/187] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:23:40 2022] GET /compute/v2.1/servers/fbef28bb-3cdf-4a27-8493-add7806af828 => generated 1314 bytes in 257 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:23:41.387756 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-cc36d2e5-1a46-46c9-9630-b65e9a176c39 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Calling method '>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:23:41.388462 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-cc36d2e5-1a46-46c9-9630-b65e9a176c39 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] [instance: 2ad09ab6-ef1f-47a2-bf44-6ae046cc8645] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:23:41.395538 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-cc36d2e5-1a46-46c9-9630-b65e9a176c39 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:23:41.400891 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-cc36d2e5-1a46-46c9-9630-b65e9a176c39 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:23:41.401742 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-cc36d2e5-1a46-46c9-9630-b65e9a176c39 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:23:41.403416 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-cc36d2e5-1a46-46c9-9630-b65e9a176c39 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:23:41.404273 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-cc36d2e5-1a46-46c9-9630-b65e9a176c39 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:23:41.406988 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-cc36d2e5-1a46-46c9-9630-b65e9a176c39 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:23:41.416317 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG dbcounter [-] [120864] Writing DB stats nova_cell1:SELECT=100 {{(pid=120864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 27 02:23:41.586486 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-cc36d2e5-1a46-46c9-9630-b65e9a176c39 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '26a86351583d49d090b3db09f127d49f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5d5ec8450c0941d18d63558baab54737', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:23:41.595611 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-cc36d2e5-1a46-46c9-9630-b65e9a176c39 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] 10.222.0.52 "GET /compute/v2.1/servers/2ad09ab6-ef1f-47a2-bf44-6ae046cc8645" status: 200 len: 1366 microversion: 2.1 time: 0.213815 Aug 27 02:23:41.596300 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 93/188] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:23:41 2022] GET /compute/v2.1/servers/2ad09ab6-ef1f-47a2-bf44-6ae046cc8645 => generated 1366 bytes in 215 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:23:41.897307 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-ccfe1d9f-35aa-4525-8ff0-1acc3c195446 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:23:41.898013 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-ccfe1d9f-35aa-4525-8ff0-1acc3c195446 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] [instance: 05bed8e9-f530-418a-9819-4f65dabdcda0] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:23:41.904386 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-ccfe1d9f-35aa-4525-8ff0-1acc3c195446 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:23:41.904999 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-ccfe1d9f-35aa-4525-8ff0-1acc3c195446 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:23:41.905685 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-ccfe1d9f-35aa-4525-8ff0-1acc3c195446 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:23:41.906604 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-ccfe1d9f-35aa-4525-8ff0-1acc3c195446 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:23:41.907186 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-ccfe1d9f-35aa-4525-8ff0-1acc3c195446 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:23:41.907831 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-ccfe1d9f-35aa-4525-8ff0-1acc3c195446 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:23:41.975842 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-90cc231e-93de-4aaa-a695-6b012662fafb tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Calling method '>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:23:41.978760 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-90cc231e-93de-4aaa-a695-6b012662fafb tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] [instance: fbef28bb-3cdf-4a27-8493-add7806af828] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:23:41.989667 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-90cc231e-93de-4aaa-a695-6b012662fafb tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:23:41.991049 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-90cc231e-93de-4aaa-a695-6b012662fafb tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:23:41.991966 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-90cc231e-93de-4aaa-a695-6b012662fafb tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:23:41.993965 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-90cc231e-93de-4aaa-a695-6b012662fafb tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:23:41.994505 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-90cc231e-93de-4aaa-a695-6b012662fafb tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:23:41.995107 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-90cc231e-93de-4aaa-a695-6b012662fafb tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:23:42.323301 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-ccfe1d9f-35aa-4525-8ff0-1acc3c195446 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd945d512ae034ec08838d0a3afde2429', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '880ac4003248441c809a76cfd1b221eb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:23:42.335109 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-ccfe1d9f-35aa-4525-8ff0-1acc3c195446 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] 10.222.0.52 "GET /compute/v2.1/servers/05bed8e9-f530-418a-9819-4f65dabdcda0" status: 200 len: 1322 microversion: 2.1 time: 0.441690 Aug 27 02:23:42.335691 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 96/189] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:23:41 2022] GET /compute/v2.1/servers/05bed8e9-f530-418a-9819-4f65dabdcda0 => generated 1322 bytes in 443 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 27 02:23:42.415287 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-90cc231e-93de-4aaa-a695-6b012662fafb tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '55640f750d1b49be9c3f79501a63c1ab', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7ef4b44a3f94d599eacce269b06594c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:23:42.423792 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-90cc231e-93de-4aaa-a695-6b012662fafb tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] 10.222.0.52 "GET /compute/v2.1/servers/fbef28bb-3cdf-4a27-8493-add7806af828" status: 200 len: 1314 microversion: 2.1 time: 0.454664 Aug 27 02:23:42.424576 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 94/190] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:23:41 2022] GET /compute/v2.1/servers/fbef28bb-3cdf-4a27-8493-add7806af828 => generated 1314 bytes in 458 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:23:42.617838 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-ca8d262e-0489-43a1-b07f-a1a6b391bbfc tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:23:42.618444 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-ca8d262e-0489-43a1-b07f-a1a6b391bbfc tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] [instance: 2ad09ab6-ef1f-47a2-bf44-6ae046cc8645] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:23:42.624604 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-ca8d262e-0489-43a1-b07f-a1a6b391bbfc tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:23:42.625076 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-ca8d262e-0489-43a1-b07f-a1a6b391bbfc tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:23:42.625943 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-ca8d262e-0489-43a1-b07f-a1a6b391bbfc tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:23:42.626421 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-ca8d262e-0489-43a1-b07f-a1a6b391bbfc tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:23:42.626865 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-ca8d262e-0489-43a1-b07f-a1a6b391bbfc tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:23:42.627421 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-ca8d262e-0489-43a1-b07f-a1a6b391bbfc tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:23:42.799929 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-ca8d262e-0489-43a1-b07f-a1a6b391bbfc tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '26a86351583d49d090b3db09f127d49f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5d5ec8450c0941d18d63558baab54737', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:23:42.808748 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-ca8d262e-0489-43a1-b07f-a1a6b391bbfc tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] 10.222.0.52 "GET /compute/v2.1/servers/2ad09ab6-ef1f-47a2-bf44-6ae046cc8645" status: 200 len: 1366 microversion: 2.1 time: 0.195421 Aug 27 02:23:42.809343 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 97/191] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:23:42 2022] GET /compute/v2.1/servers/2ad09ab6-ef1f-47a2-bf44-6ae046cc8645 => generated 1366 bytes in 197 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:23:43.374976 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-207684b4-b7b7-4652-b2c5-1801129c63b8 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Calling method '>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:23:43.377416 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-207684b4-b7b7-4652-b2c5-1801129c63b8 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] [instance: 05bed8e9-f530-418a-9819-4f65dabdcda0] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:23:43.385385 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-207684b4-b7b7-4652-b2c5-1801129c63b8 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:23:43.385975 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-207684b4-b7b7-4652-b2c5-1801129c63b8 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:23:43.386460 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-207684b4-b7b7-4652-b2c5-1801129c63b8 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:23:43.387276 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-207684b4-b7b7-4652-b2c5-1801129c63b8 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:23:43.387785 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-207684b4-b7b7-4652-b2c5-1801129c63b8 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:23:43.388365 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-207684b4-b7b7-4652-b2c5-1801129c63b8 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:23:43.448088 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-8ef8fd5a-bb3a-4a01-a653-fb4f1c543489 tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:23:43.450245 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-8ef8fd5a-bb3a-4a01-a653-fb4f1c543489 tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] [instance: fbef28bb-3cdf-4a27-8493-add7806af828] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:23:43.457762 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-8ef8fd5a-bb3a-4a01-a653-fb4f1c543489 tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:23:43.458463 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-8ef8fd5a-bb3a-4a01-a653-fb4f1c543489 tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:23:43.459332 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-8ef8fd5a-bb3a-4a01-a653-fb4f1c543489 tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:23:43.460382 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-8ef8fd5a-bb3a-4a01-a653-fb4f1c543489 tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:23:43.461642 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-8ef8fd5a-bb3a-4a01-a653-fb4f1c543489 tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:23:43.462320 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-8ef8fd5a-bb3a-4a01-a653-fb4f1c543489 tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:23:43.643044 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-207684b4-b7b7-4652-b2c5-1801129c63b8 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd945d512ae034ec08838d0a3afde2429', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '880ac4003248441c809a76cfd1b221eb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:23:43.657003 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-207684b4-b7b7-4652-b2c5-1801129c63b8 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] 10.222.0.52 "GET /compute/v2.1/servers/05bed8e9-f530-418a-9819-4f65dabdcda0" status: 200 len: 1322 microversion: 2.1 time: 0.287099 Aug 27 02:23:43.658517 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 95/192] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:23:43 2022] GET /compute/v2.1/servers/05bed8e9-f530-418a-9819-4f65dabdcda0 => generated 1322 bytes in 290 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:23:43.683504 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-8ef8fd5a-bb3a-4a01-a653-fb4f1c543489 tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '55640f750d1b49be9c3f79501a63c1ab', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7ef4b44a3f94d599eacce269b06594c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:23:43.693521 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-8ef8fd5a-bb3a-4a01-a653-fb4f1c543489 tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] 10.222.0.52 "GET /compute/v2.1/servers/fbef28bb-3cdf-4a27-8493-add7806af828" status: 200 len: 1314 microversion: 2.1 time: 0.249850 Aug 27 02:23:43.694184 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 98/193] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:23:43 2022] GET /compute/v2.1/servers/fbef28bb-3cdf-4a27-8493-add7806af828 => generated 1314 bytes in 251 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 27 02:23:43.834575 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-745b14b8-02fa-4cd7-af61-2b9d62ab6bc5 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Calling method '>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:23:43.835179 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-745b14b8-02fa-4cd7-af61-2b9d62ab6bc5 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] [instance: 2ad09ab6-ef1f-47a2-bf44-6ae046cc8645] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:23:43.843272 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-745b14b8-02fa-4cd7-af61-2b9d62ab6bc5 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:23:43.843757 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-745b14b8-02fa-4cd7-af61-2b9d62ab6bc5 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:23:43.844323 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-745b14b8-02fa-4cd7-af61-2b9d62ab6bc5 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:23:43.845242 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-745b14b8-02fa-4cd7-af61-2b9d62ab6bc5 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:23:43.845696 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-745b14b8-02fa-4cd7-af61-2b9d62ab6bc5 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:23:43.846115 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-745b14b8-02fa-4cd7-af61-2b9d62ab6bc5 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:23:44.083089 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-745b14b8-02fa-4cd7-af61-2b9d62ab6bc5 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '26a86351583d49d090b3db09f127d49f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5d5ec8450c0941d18d63558baab54737', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:23:44.102666 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-745b14b8-02fa-4cd7-af61-2b9d62ab6bc5 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] 10.222.0.52 "GET /compute/v2.1/servers/2ad09ab6-ef1f-47a2-bf44-6ae046cc8645" status: 200 len: 1366 microversion: 2.1 time: 0.272159 Aug 27 02:23:44.103689 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 96/194] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:23:43 2022] GET /compute/v2.1/servers/2ad09ab6-ef1f-47a2-bf44-6ae046cc8645 => generated 1366 bytes in 274 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:23:44.681872 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-e8aceaa2-8b5e-4a11-b383-484d1630f627 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:23:44.683101 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-e8aceaa2-8b5e-4a11-b383-484d1630f627 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] [instance: 05bed8e9-f530-418a-9819-4f65dabdcda0] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:23:44.693547 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-e8aceaa2-8b5e-4a11-b383-484d1630f627 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:23:44.694407 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-e8aceaa2-8b5e-4a11-b383-484d1630f627 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:23:44.695805 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-e8aceaa2-8b5e-4a11-b383-484d1630f627 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:23:44.697031 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-e8aceaa2-8b5e-4a11-b383-484d1630f627 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:23:44.697897 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-e8aceaa2-8b5e-4a11-b383-484d1630f627 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:23:44.699005 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-e8aceaa2-8b5e-4a11-b383-484d1630f627 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:23:44.719445 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-b61ebe13-8add-4a28-b0cb-4ec7b5d7e07d tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Calling method '>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:23:44.721192 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-b61ebe13-8add-4a28-b0cb-4ec7b5d7e07d tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] [instance: fbef28bb-3cdf-4a27-8493-add7806af828] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:23:44.744209 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-b61ebe13-8add-4a28-b0cb-4ec7b5d7e07d tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:23:44.744908 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-b61ebe13-8add-4a28-b0cb-4ec7b5d7e07d tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:23:44.745396 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-b61ebe13-8add-4a28-b0cb-4ec7b5d7e07d tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:23:44.746101 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-b61ebe13-8add-4a28-b0cb-4ec7b5d7e07d tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:23:44.746543 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-b61ebe13-8add-4a28-b0cb-4ec7b5d7e07d tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:23:44.746965 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-b61ebe13-8add-4a28-b0cb-4ec7b5d7e07d tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:23:44.998295 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-e8aceaa2-8b5e-4a11-b383-484d1630f627 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd945d512ae034ec08838d0a3afde2429', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '880ac4003248441c809a76cfd1b221eb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:23:45.013432 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-e8aceaa2-8b5e-4a11-b383-484d1630f627 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] 10.222.0.52 "GET /compute/v2.1/servers/05bed8e9-f530-418a-9819-4f65dabdcda0" status: 200 len: 1322 microversion: 2.1 time: 0.335312 Aug 27 02:23:45.028965 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 99/195] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:23:44 2022] GET /compute/v2.1/servers/05bed8e9-f530-418a-9819-4f65dabdcda0 => generated 1322 bytes in 352 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:23:45.035409 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [req-5f12596a-461a-437a-916c-3cec8424870f req-7dd1476a-00a1-4ec9-a35a-f843706e1315 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "05bed8e9-f530-418a-9819-4f65dabdcda0", "name": "network-vif-plugged", "status": "completed", "tag": "81921b15-5b92-41f8-a6fa-30be932b9435"}]} {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 27 02:23:45.046986 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [req-5f12596a-461a-437a-916c-3cec8424870f req-7dd1476a-00a1-4ec9-a35a-f843706e1315 service nova] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:23:45.047784 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [req-5f12596a-461a-437a-916c-3cec8424870f req-7dd1476a-00a1-4ec9-a35a-f843706e1315 service nova] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:23:45.048720 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [req-5f12596a-461a-437a-916c-3cec8424870f req-7dd1476a-00a1-4ec9-a35a-f843706e1315 service nova] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:23:45.066049 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.compute.server_external_events [req-5f12596a-461a-437a-916c-3cec8424870f req-7dd1476a-00a1-4ec9-a35a-f843706e1315 service nova] Creating event network-vif-plugged:81921b15-5b92-41f8-a6fa-30be932b9435 for instance 05bed8e9-f530-418a-9819-4f65dabdcda0 on hv-cinder-84630 Aug 27 02:23:45.074160 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [req-5f12596a-461a-437a-916c-3cec8424870f req-7dd1476a-00a1-4ec9-a35a-f843706e1315 service nova] 10.222.0.52 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.042922 Aug 27 02:23:45.074985 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 100/196] 10.222.0.52 () {64 vars in 1317 bytes} [Sat Aug 27 02:23:45 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 27 02:23:45.116522 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-b61ebe13-8add-4a28-b0cb-4ec7b5d7e07d tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '55640f750d1b49be9c3f79501a63c1ab', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7ef4b44a3f94d599eacce269b06594c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:23:45.128023 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-b61ebe13-8add-4a28-b0cb-4ec7b5d7e07d tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] 10.222.0.52 "GET /compute/v2.1/servers/fbef28bb-3cdf-4a27-8493-add7806af828" status: 200 len: 1314 microversion: 2.1 time: 0.415432 Aug 27 02:23:45.128670 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 97/197] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:23:44 2022] GET /compute/v2.1/servers/fbef28bb-3cdf-4a27-8493-add7806af828 => generated 1314 bytes in 417 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 27 02:23:45.128757 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-b426deb0-55ec-49c6-ac7c-c45aca8053ce tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:23:45.130272 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-b426deb0-55ec-49c6-ac7c-c45aca8053ce tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] [instance: 2ad09ab6-ef1f-47a2-bf44-6ae046cc8645] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:23:45.149600 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-b426deb0-55ec-49c6-ac7c-c45aca8053ce tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:23:45.150267 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-b426deb0-55ec-49c6-ac7c-c45aca8053ce tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:23:45.150779 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-b426deb0-55ec-49c6-ac7c-c45aca8053ce tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:23:45.151514 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-b426deb0-55ec-49c6-ac7c-c45aca8053ce tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:23:45.151956 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-b426deb0-55ec-49c6-ac7c-c45aca8053ce tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:23:45.152585 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-b426deb0-55ec-49c6-ac7c-c45aca8053ce tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:23:45.324233 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-b426deb0-55ec-49c6-ac7c-c45aca8053ce tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '26a86351583d49d090b3db09f127d49f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5d5ec8450c0941d18d63558baab54737', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:23:45.345410 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-b426deb0-55ec-49c6-ac7c-c45aca8053ce tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] 10.222.0.52 "GET /compute/v2.1/servers/2ad09ab6-ef1f-47a2-bf44-6ae046cc8645" status: 200 len: 1366 microversion: 2.1 time: 0.221841 Aug 27 02:23:45.346076 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 101/198] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:23:45 2022] GET /compute/v2.1/servers/2ad09ab6-ef1f-47a2-bf44-6ae046cc8645 => generated 1366 bytes in 224 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:23:46.041271 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-6f4dc362-a9c8-4198-9f22-f78b543dc4c7 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Calling method '>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:23:46.041882 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-6f4dc362-a9c8-4198-9f22-f78b543dc4c7 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] [instance: 05bed8e9-f530-418a-9819-4f65dabdcda0] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:23:46.054066 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-6f4dc362-a9c8-4198-9f22-f78b543dc4c7 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:23:46.055130 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-6f4dc362-a9c8-4198-9f22-f78b543dc4c7 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:23:46.055995 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-6f4dc362-a9c8-4198-9f22-f78b543dc4c7 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:23:46.057039 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-6f4dc362-a9c8-4198-9f22-f78b543dc4c7 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:23:46.058070 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-6f4dc362-a9c8-4198-9f22-f78b543dc4c7 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:23:46.058756 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-6f4dc362-a9c8-4198-9f22-f78b543dc4c7 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:23:46.152491 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-d246c993-033d-4219-99cd-80778d4565d9 tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:23:46.153358 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-d246c993-033d-4219-99cd-80778d4565d9 tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] [instance: fbef28bb-3cdf-4a27-8493-add7806af828] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:23:46.160232 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-d246c993-033d-4219-99cd-80778d4565d9 tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:23:46.162567 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-d246c993-033d-4219-99cd-80778d4565d9 tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:23:46.163834 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-d246c993-033d-4219-99cd-80778d4565d9 tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:23:46.165013 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-d246c993-033d-4219-99cd-80778d4565d9 tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:23:46.165977 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-d246c993-033d-4219-99cd-80778d4565d9 tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:23:46.166585 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-d246c993-033d-4219-99cd-80778d4565d9 tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:23:46.297325 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-6f4dc362-a9c8-4198-9f22-f78b543dc4c7 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd945d512ae034ec08838d0a3afde2429', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '880ac4003248441c809a76cfd1b221eb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:23:46.306384 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-6f4dc362-a9c8-4198-9f22-f78b543dc4c7 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] 10.222.0.52 "GET /compute/v2.1/servers/05bed8e9-f530-418a-9819-4f65dabdcda0" status: 200 len: 1322 microversion: 2.1 time: 0.269467 Aug 27 02:23:46.319470 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 98/199] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:23:46 2022] GET /compute/v2.1/servers/05bed8e9-f530-418a-9819-4f65dabdcda0 => generated 1322 bytes in 283 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:23:46.369415 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-a9a77bfb-5951-47cd-91be-cce1ce3d4e84 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Calling method '>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:23:46.371227 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-a9a77bfb-5951-47cd-91be-cce1ce3d4e84 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] [instance: 2ad09ab6-ef1f-47a2-bf44-6ae046cc8645] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:23:46.392665 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-a9a77bfb-5951-47cd-91be-cce1ce3d4e84 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:23:46.395342 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-a9a77bfb-5951-47cd-91be-cce1ce3d4e84 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:23:46.398038 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-a9a77bfb-5951-47cd-91be-cce1ce3d4e84 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:23:46.400100 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-a9a77bfb-5951-47cd-91be-cce1ce3d4e84 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:23:46.402393 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-a9a77bfb-5951-47cd-91be-cce1ce3d4e84 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:23:46.403251 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-a9a77bfb-5951-47cd-91be-cce1ce3d4e84 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:23:46.410076 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-d246c993-033d-4219-99cd-80778d4565d9 tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '55640f750d1b49be9c3f79501a63c1ab', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7ef4b44a3f94d599eacce269b06594c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:23:46.420453 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-d246c993-033d-4219-99cd-80778d4565d9 tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] 10.222.0.52 "GET /compute/v2.1/servers/fbef28bb-3cdf-4a27-8493-add7806af828" status: 200 len: 1314 microversion: 2.1 time: 0.272489 Aug 27 02:23:46.424069 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 102/200] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:23:46 2022] GET /compute/v2.1/servers/fbef28bb-3cdf-4a27-8493-add7806af828 => generated 1314 bytes in 277 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 27 02:23:46.795859 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-a9a77bfb-5951-47cd-91be-cce1ce3d4e84 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '26a86351583d49d090b3db09f127d49f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5d5ec8450c0941d18d63558baab54737', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:23:46.805702 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-a9a77bfb-5951-47cd-91be-cce1ce3d4e84 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] 10.222.0.52 "GET /compute/v2.1/servers/2ad09ab6-ef1f-47a2-bf44-6ae046cc8645" status: 200 len: 1366 microversion: 2.1 time: 0.442892 Aug 27 02:23:46.814670 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 99/201] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:23:46 2022] GET /compute/v2.1/servers/2ad09ab6-ef1f-47a2-bf44-6ae046cc8645 => generated 1366 bytes in 452 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:23:47.363271 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-7583fc58-9ff7-483a-a427-64a27ae9a537 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:23:47.365788 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-7583fc58-9ff7-483a-a427-64a27ae9a537 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] [instance: 05bed8e9-f530-418a-9819-4f65dabdcda0] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:23:47.372761 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-7583fc58-9ff7-483a-a427-64a27ae9a537 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:23:47.373681 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-7583fc58-9ff7-483a-a427-64a27ae9a537 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:23:47.374321 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-7583fc58-9ff7-483a-a427-64a27ae9a537 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:23:47.375289 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-7583fc58-9ff7-483a-a427-64a27ae9a537 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:23:47.375766 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-7583fc58-9ff7-483a-a427-64a27ae9a537 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:23:47.376378 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-7583fc58-9ff7-483a-a427-64a27ae9a537 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:23:47.381836 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG dbcounter [-] [120863] Writing DB stats nova_cell1:SELECT=100 {{(pid=120863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 27 02:23:47.457386 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-4888371f-6008-4611-abb7-b54ab12b87d4 tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Calling method '>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:23:47.458272 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-4888371f-6008-4611-abb7-b54ab12b87d4 tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] [instance: fbef28bb-3cdf-4a27-8493-add7806af828] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:23:47.471341 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-4888371f-6008-4611-abb7-b54ab12b87d4 tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:23:47.472738 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-4888371f-6008-4611-abb7-b54ab12b87d4 tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:23:47.473418 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-4888371f-6008-4611-abb7-b54ab12b87d4 tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:23:47.474506 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-4888371f-6008-4611-abb7-b54ab12b87d4 tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:23:47.475930 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-4888371f-6008-4611-abb7-b54ab12b87d4 tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:23:47.476781 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-4888371f-6008-4611-abb7-b54ab12b87d4 tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:23:47.564226 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-7583fc58-9ff7-483a-a427-64a27ae9a537 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd945d512ae034ec08838d0a3afde2429', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '880ac4003248441c809a76cfd1b221eb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:23:47.578427 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-7583fc58-9ff7-483a-a427-64a27ae9a537 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] 10.222.0.52 "GET /compute/v2.1/servers/05bed8e9-f530-418a-9819-4f65dabdcda0" status: 200 len: 1322 microversion: 2.1 time: 0.222054 Aug 27 02:23:47.579200 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 103/202] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:23:47 2022] GET /compute/v2.1/servers/05bed8e9-f530-418a-9819-4f65dabdcda0 => generated 1322 bytes in 225 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:23:47.681622 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-4888371f-6008-4611-abb7-b54ab12b87d4 tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '55640f750d1b49be9c3f79501a63c1ab', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7ef4b44a3f94d599eacce269b06594c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:23:47.707923 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-4888371f-6008-4611-abb7-b54ab12b87d4 tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] 10.222.0.52 "GET /compute/v2.1/servers/fbef28bb-3cdf-4a27-8493-add7806af828" status: 200 len: 1314 microversion: 2.1 time: 0.254506 Aug 27 02:23:47.709450 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 100/203] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:23:47 2022] GET /compute/v2.1/servers/fbef28bb-3cdf-4a27-8493-add7806af828 => generated 1314 bytes in 257 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:23:47.855779 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-8a5443a5-06de-4db0-a1df-ddde37477171 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:23:47.856384 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-8a5443a5-06de-4db0-a1df-ddde37477171 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] [instance: 2ad09ab6-ef1f-47a2-bf44-6ae046cc8645] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:23:47.863410 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-8a5443a5-06de-4db0-a1df-ddde37477171 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:23:47.863939 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-8a5443a5-06de-4db0-a1df-ddde37477171 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:23:47.864525 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-8a5443a5-06de-4db0-a1df-ddde37477171 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:23:47.865770 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-8a5443a5-06de-4db0-a1df-ddde37477171 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:23:47.866430 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-8a5443a5-06de-4db0-a1df-ddde37477171 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:23:47.867782 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-8a5443a5-06de-4db0-a1df-ddde37477171 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:23:48.102821 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-8a5443a5-06de-4db0-a1df-ddde37477171 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '26a86351583d49d090b3db09f127d49f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5d5ec8450c0941d18d63558baab54737', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:23:48.113375 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-8a5443a5-06de-4db0-a1df-ddde37477171 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] 10.222.0.52 "GET /compute/v2.1/servers/2ad09ab6-ef1f-47a2-bf44-6ae046cc8645" status: 200 len: 1366 microversion: 2.1 time: 0.265487 Aug 27 02:23:48.117202 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 104/204] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:23:47 2022] GET /compute/v2.1/servers/2ad09ab6-ef1f-47a2-bf44-6ae046cc8645 => generated 1366 bytes in 270 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:23:48.599875 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-96cd5c91-f9db-4aba-9623-c17c72436912 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Calling method '>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:23:48.600517 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-96cd5c91-f9db-4aba-9623-c17c72436912 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] [instance: 05bed8e9-f530-418a-9819-4f65dabdcda0] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:23:48.606529 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-96cd5c91-f9db-4aba-9623-c17c72436912 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:23:48.607960 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-96cd5c91-f9db-4aba-9623-c17c72436912 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:23:48.608340 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-96cd5c91-f9db-4aba-9623-c17c72436912 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:23:48.609065 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-96cd5c91-f9db-4aba-9623-c17c72436912 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:23:48.609530 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-96cd5c91-f9db-4aba-9623-c17c72436912 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:23:48.610176 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-96cd5c91-f9db-4aba-9623-c17c72436912 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:23:48.745244 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-e902b84e-783f-436a-bd3e-054565b70435 tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:23:48.747105 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-e902b84e-783f-436a-bd3e-054565b70435 tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] [instance: fbef28bb-3cdf-4a27-8493-add7806af828] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:23:48.754135 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-96cd5c91-f9db-4aba-9623-c17c72436912 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd945d512ae034ec08838d0a3afde2429', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '880ac4003248441c809a76cfd1b221eb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:23:48.760600 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-e902b84e-783f-436a-bd3e-054565b70435 tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:23:48.761425 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-e902b84e-783f-436a-bd3e-054565b70435 tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:23:48.763958 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-e902b84e-783f-436a-bd3e-054565b70435 tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:23:48.764921 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-e902b84e-783f-436a-bd3e-054565b70435 tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:23:48.767484 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-96cd5c91-f9db-4aba-9623-c17c72436912 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] 10.222.0.52 "GET /compute/v2.1/servers/05bed8e9-f530-418a-9819-4f65dabdcda0" status: 200 len: 1322 microversion: 2.1 time: 0.172038 Aug 27 02:23:48.767892 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-e902b84e-783f-436a-bd3e-054565b70435 tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:23:48.768292 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 101/205] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:23:48 2022] GET /compute/v2.1/servers/05bed8e9-f530-418a-9819-4f65dabdcda0 => generated 1322 bytes in 174 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 27 02:23:48.768448 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-e902b84e-783f-436a-bd3e-054565b70435 tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:23:48.924184 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-e902b84e-783f-436a-bd3e-054565b70435 tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '55640f750d1b49be9c3f79501a63c1ab', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7ef4b44a3f94d599eacce269b06594c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:23:48.933726 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-e902b84e-783f-436a-bd3e-054565b70435 tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] 10.222.0.52 "GET /compute/v2.1/servers/fbef28bb-3cdf-4a27-8493-add7806af828" status: 200 len: 1314 microversion: 2.1 time: 0.206877 Aug 27 02:23:48.934307 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 105/206] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:23:48 2022] GET /compute/v2.1/servers/fbef28bb-3cdf-4a27-8493-add7806af828 => generated 1314 bytes in 209 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 27 02:23:49.148370 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-0210a7f2-8df0-431a-99ea-155b92f73c6c tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Calling method '>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:23:49.149298 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-0210a7f2-8df0-431a-99ea-155b92f73c6c tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] [instance: 2ad09ab6-ef1f-47a2-bf44-6ae046cc8645] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:23:49.158870 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-0210a7f2-8df0-431a-99ea-155b92f73c6c tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:23:49.159618 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-0210a7f2-8df0-431a-99ea-155b92f73c6c tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:23:49.160314 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-0210a7f2-8df0-431a-99ea-155b92f73c6c tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:23:49.161714 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-0210a7f2-8df0-431a-99ea-155b92f73c6c tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:23:49.162831 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-0210a7f2-8df0-431a-99ea-155b92f73c6c tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:23:49.163519 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-0210a7f2-8df0-431a-99ea-155b92f73c6c tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:23:49.296723 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [req-6602d749-1d3c-4b44-8e4a-db336b51a5a9 req-4224f00c-49bf-4c17-94a6-9b9af3aedbe7 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "e2bf7024-0779-4e08-844a-43b78187e770", "name": "network-vif-plugged", "status": "completed", "tag": "66f55f99-dc75-425b-9214-a2edc384e389"}]} {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 27 02:23:49.313498 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [req-6602d749-1d3c-4b44-8e4a-db336b51a5a9 req-4224f00c-49bf-4c17-94a6-9b9af3aedbe7 service nova] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:23:49.314097 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [req-6602d749-1d3c-4b44-8e4a-db336b51a5a9 req-4224f00c-49bf-4c17-94a6-9b9af3aedbe7 service nova] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:23:49.318146 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [req-6602d749-1d3c-4b44-8e4a-db336b51a5a9 req-4224f00c-49bf-4c17-94a6-9b9af3aedbe7 service nova] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:23:49.339908 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.compute.server_external_events [req-6602d749-1d3c-4b44-8e4a-db336b51a5a9 req-4224f00c-49bf-4c17-94a6-9b9af3aedbe7 service nova] Creating event network-vif-plugged:66f55f99-dc75-425b-9214-a2edc384e389 for instance e2bf7024-0779-4e08-844a-43b78187e770 on hv-cinder-84630 Aug 27 02:23:49.345844 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [req-6602d749-1d3c-4b44-8e4a-db336b51a5a9 req-4224f00c-49bf-4c17-94a6-9b9af3aedbe7 service nova] 10.222.0.52 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.064932 Aug 27 02:23:49.346786 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 106/207] 10.222.0.52 () {64 vars in 1317 bytes} [Sat Aug 27 02:23:49 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 66 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Aug 27 02:23:49.355822 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-0210a7f2-8df0-431a-99ea-155b92f73c6c tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '26a86351583d49d090b3db09f127d49f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5d5ec8450c0941d18d63558baab54737', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:23:49.371167 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-0210a7f2-8df0-431a-99ea-155b92f73c6c tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] 10.222.0.52 "GET /compute/v2.1/servers/2ad09ab6-ef1f-47a2-bf44-6ae046cc8645" status: 200 len: 1555 microversion: 2.1 time: 0.228455 Aug 27 02:23:49.372030 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 102/208] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:23:49 2022] GET /compute/v2.1/servers/2ad09ab6-ef1f-47a2-bf44-6ae046cc8645 => generated 1555 bytes in 230 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:23:49.396023 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-37233d77-ac79-4caa-9966-2ff1e903d9af tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:23:49.396590 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-37233d77-ac79-4caa-9966-2ff1e903d9af tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] [instance: 2ad09ab6-ef1f-47a2-bf44-6ae046cc8645] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:23:49.403849 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-37233d77-ac79-4caa-9966-2ff1e903d9af tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:23:49.404332 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-37233d77-ac79-4caa-9966-2ff1e903d9af tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:23:49.404862 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-37233d77-ac79-4caa-9966-2ff1e903d9af tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:23:49.406329 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-37233d77-ac79-4caa-9966-2ff1e903d9af tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:23:49.406329 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-37233d77-ac79-4caa-9966-2ff1e903d9af tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:23:49.406916 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-37233d77-ac79-4caa-9966-2ff1e903d9af tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:23:49.539346 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-37233d77-ac79-4caa-9966-2ff1e903d9af tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] 10.222.0.52 "GET /compute/v2.1/servers/2ad09ab6-ef1f-47a2-bf44-6ae046cc8645/os-interface" status: 200 len: 284 microversion: 2.1 time: 0.148567 Aug 27 02:23:49.540389 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 107/209] 10.222.0.52 () {58 vars in 1275 bytes} [Sat Aug 27 02:23:49 2022] GET /compute/v2.1/servers/2ad09ab6-ef1f-47a2-bf44-6ae046cc8645/os-interface => generated 284 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 27 02:23:49.789893 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-10b8036e-91ef-4271-add2-ef5855c7456e tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Calling method '>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:23:49.790721 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-10b8036e-91ef-4271-add2-ef5855c7456e tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] [instance: 05bed8e9-f530-418a-9819-4f65dabdcda0] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:23:49.796846 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-10b8036e-91ef-4271-add2-ef5855c7456e tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:23:49.797473 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-10b8036e-91ef-4271-add2-ef5855c7456e tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:23:49.797914 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-10b8036e-91ef-4271-add2-ef5855c7456e tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:23:49.798600 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-10b8036e-91ef-4271-add2-ef5855c7456e tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:23:49.799045 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-10b8036e-91ef-4271-add2-ef5855c7456e tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:23:49.799460 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-10b8036e-91ef-4271-add2-ef5855c7456e tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:23:49.955457 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-6956a537-f437-429a-817f-df3ac713bf84 tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:23:49.956251 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-6956a537-f437-429a-817f-df3ac713bf84 tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] [instance: fbef28bb-3cdf-4a27-8493-add7806af828] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:23:49.962895 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-6956a537-f437-429a-817f-df3ac713bf84 tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:23:49.963420 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-6956a537-f437-429a-817f-df3ac713bf84 tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:23:49.963849 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-6956a537-f437-429a-817f-df3ac713bf84 tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:23:49.964845 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-6956a537-f437-429a-817f-df3ac713bf84 tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:23:49.965335 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-6956a537-f437-429a-817f-df3ac713bf84 tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:23:49.965763 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-6956a537-f437-429a-817f-df3ac713bf84 tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:23:50.162278 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-10b8036e-91ef-4271-add2-ef5855c7456e tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd945d512ae034ec08838d0a3afde2429', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '880ac4003248441c809a76cfd1b221eb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:23:50.171568 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-10b8036e-91ef-4271-add2-ef5855c7456e tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] 10.222.0.52 "GET /compute/v2.1/servers/05bed8e9-f530-418a-9819-4f65dabdcda0" status: 200 len: 1322 microversion: 2.1 time: 0.386081 Aug 27 02:23:50.172587 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 103/210] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:23:49 2022] GET /compute/v2.1/servers/05bed8e9-f530-418a-9819-4f65dabdcda0 => generated 1322 bytes in 388 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:23:50.585400 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-6956a537-f437-429a-817f-df3ac713bf84 tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '55640f750d1b49be9c3f79501a63c1ab', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7ef4b44a3f94d599eacce269b06594c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:23:50.606643 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-6956a537-f437-429a-817f-df3ac713bf84 tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] 10.222.0.52 "GET /compute/v2.1/servers/fbef28bb-3cdf-4a27-8493-add7806af828" status: 200 len: 1502 microversion: 2.1 time: 0.654228 Aug 27 02:23:50.607249 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 108/211] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:23:49 2022] GET /compute/v2.1/servers/fbef28bb-3cdf-4a27-8493-add7806af828 => generated 1502 bytes in 656 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 27 02:23:51.196444 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-ba393b09-2340-4491-bb58-3f5685f6d9cd tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Calling method '>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:23:51.197109 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-ba393b09-2340-4491-bb58-3f5685f6d9cd tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] [instance: 05bed8e9-f530-418a-9819-4f65dabdcda0] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:23:51.218549 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-ba393b09-2340-4491-bb58-3f5685f6d9cd tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:23:51.219564 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-ba393b09-2340-4491-bb58-3f5685f6d9cd tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:23:51.220391 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-ba393b09-2340-4491-bb58-3f5685f6d9cd tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:23:51.221509 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-ba393b09-2340-4491-bb58-3f5685f6d9cd tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:23:51.222239 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-ba393b09-2340-4491-bb58-3f5685f6d9cd tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:23:51.222902 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-ba393b09-2340-4491-bb58-3f5685f6d9cd tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:23:51.439242 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-ba393b09-2340-4491-bb58-3f5685f6d9cd tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd945d512ae034ec08838d0a3afde2429', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '880ac4003248441c809a76cfd1b221eb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:23:51.457025 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-ba393b09-2340-4491-bb58-3f5685f6d9cd tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] 10.222.0.52 "GET /compute/v2.1/servers/05bed8e9-f530-418a-9819-4f65dabdcda0" status: 200 len: 1515 microversion: 2.1 time: 0.265300 Aug 27 02:23:51.470438 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 104/212] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:23:51 2022] GET /compute/v2.1/servers/05bed8e9-f530-418a-9819-4f65dabdcda0 => generated 1515 bytes in 279 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:23:51.482694 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-2669f840-4043-4ecd-88fe-24ba24cef7f5 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Action: 'action', calling method: >, body: {"rescue": {"adminPass": "***"}} {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 27 02:23:51.484926 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-2669f840-4043-4ecd-88fe-24ba24cef7f5 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] [instance: 05bed8e9-f530-418a-9819-4f65dabdcda0] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:23:51.500997 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-2669f840-4043-4ecd-88fe-24ba24cef7f5 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:23:51.501512 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-2669f840-4043-4ecd-88fe-24ba24cef7f5 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:23:51.502592 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-2669f840-4043-4ecd-88fe-24ba24cef7f5 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:23:51.503533 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-2669f840-4043-4ecd-88fe-24ba24cef7f5 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:23:51.504575 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-2669f840-4043-4ecd-88fe-24ba24cef7f5 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:23:51.505071 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-2669f840-4043-4ecd-88fe-24ba24cef7f5 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:23:51.532226 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.objects.instance [None req-2669f840-4043-4ecd-88fe-24ba24cef7f5 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lazy-loading 'flavor' on Instance uuid 05bed8e9-f530-418a-9819-4f65dabdcda0 {{(pid=120863) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Aug 27 02:23:51.657799 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [req-c7f4bea0-bc2d-4c9e-8040-36dea70c5384 req-a646cfcf-7928-42bd-afab-b0417f7cc1ba service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "2ad09ab6-ef1f-47a2-bf44-6ae046cc8645", "tag": "e98d7081-9ec1-49da-9111-4786cc0b96ef"}]} {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 27 02:23:51.674128 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [req-c7f4bea0-bc2d-4c9e-8040-36dea70c5384 req-a646cfcf-7928-42bd-afab-b0417f7cc1ba service nova] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:23:51.675122 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [req-c7f4bea0-bc2d-4c9e-8040-36dea70c5384 req-a646cfcf-7928-42bd-afab-b0417f7cc1ba service nova] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:23:51.676009 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [req-c7f4bea0-bc2d-4c9e-8040-36dea70c5384 req-a646cfcf-7928-42bd-afab-b0417f7cc1ba service nova] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:23:51.713272 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.compute.server_external_events [req-c7f4bea0-bc2d-4c9e-8040-36dea70c5384 req-a646cfcf-7928-42bd-afab-b0417f7cc1ba service nova] Creating event network-changed:e98d7081-9ec1-49da-9111-4786cc0b96ef for instance 2ad09ab6-ef1f-47a2-bf44-6ae046cc8645 on hv-cinder-84630 Aug 27 02:23:51.729650 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [req-c7f4bea0-bc2d-4c9e-8040-36dea70c5384 req-a646cfcf-7928-42bd-afab-b0417f7cc1ba service nova] 10.222.0.52 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.077820 Aug 27 02:23:51.733078 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 105/213] 10.222.0.52 () {64 vars in 1317 bytes} [Sat Aug 27 02:23:51 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 83 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 27 02:23:51.743516 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-2669f840-4043-4ecd-88fe-24ba24cef7f5 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] 10.222.0.52 "POST /compute/v2.1/servers/05bed8e9-f530-418a-9819-4f65dabdcda0/action" status: 200 len: 32 microversion: 2.1 time: 0.265898 Aug 27 02:23:51.748075 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 109/214] 10.222.0.52 () {60 vars in 1278 bytes} [Sat Aug 27 02:23:51 2022] POST /compute/v2.1/servers/05bed8e9-f530-418a-9819-4f65dabdcda0/action => generated 32 bytes in 271 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Aug 27 02:23:51.765351 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-e74843e5-9e39-4f87-96f1-26e1b87dbb4f tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Calling method '>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:23:51.766694 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-e74843e5-9e39-4f87-96f1-26e1b87dbb4f tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] [instance: 05bed8e9-f530-418a-9819-4f65dabdcda0] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:23:51.776834 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-e74843e5-9e39-4f87-96f1-26e1b87dbb4f tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:23:51.777982 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-e74843e5-9e39-4f87-96f1-26e1b87dbb4f tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:23:51.779057 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-e74843e5-9e39-4f87-96f1-26e1b87dbb4f tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:23:51.780014 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-e74843e5-9e39-4f87-96f1-26e1b87dbb4f tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:23:51.780667 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-e74843e5-9e39-4f87-96f1-26e1b87dbb4f tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:23:51.781959 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-e74843e5-9e39-4f87-96f1-26e1b87dbb4f tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:23:52.252902 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-e74843e5-9e39-4f87-96f1-26e1b87dbb4f tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd945d512ae034ec08838d0a3afde2429', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '880ac4003248441c809a76cfd1b221eb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:23:52.266232 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-e74843e5-9e39-4f87-96f1-26e1b87dbb4f tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] 10.222.0.52 "GET /compute/v2.1/servers/05bed8e9-f530-418a-9819-4f65dabdcda0" status: 200 len: 1521 microversion: 2.1 time: 0.504224 Aug 27 02:23:52.266912 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 106/215] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:23:51 2022] GET /compute/v2.1/servers/05bed8e9-f530-418a-9819-4f65dabdcda0 => generated 1521 bytes in 510 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:23:53.302442 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-4b8fb0f3-a82f-45c9-95dd-c98b7d3cdafa tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:23:53.302442 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-4b8fb0f3-a82f-45c9-95dd-c98b7d3cdafa tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] [instance: 05bed8e9-f530-418a-9819-4f65dabdcda0] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:23:53.304558 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG dbcounter [-] [120863] Writing DB stats nova_api:SELECT=76 {{(pid=120863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 27 02:23:53.319492 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-4b8fb0f3-a82f-45c9-95dd-c98b7d3cdafa tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:23:53.320726 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-4b8fb0f3-a82f-45c9-95dd-c98b7d3cdafa tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:23:53.321277 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-4b8fb0f3-a82f-45c9-95dd-c98b7d3cdafa tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:23:53.322111 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-4b8fb0f3-a82f-45c9-95dd-c98b7d3cdafa tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:23:53.322622 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-4b8fb0f3-a82f-45c9-95dd-c98b7d3cdafa tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:23:53.323133 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-4b8fb0f3-a82f-45c9-95dd-c98b7d3cdafa tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:23:53.580761 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-4b8fb0f3-a82f-45c9-95dd-c98b7d3cdafa tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd945d512ae034ec08838d0a3afde2429', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '880ac4003248441c809a76cfd1b221eb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:23:53.597106 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-4b8fb0f3-a82f-45c9-95dd-c98b7d3cdafa tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] 10.222.0.52 "GET /compute/v2.1/servers/05bed8e9-f530-418a-9819-4f65dabdcda0" status: 200 len: 1521 microversion: 2.1 time: 0.303352 Aug 27 02:23:53.598911 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 110/216] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:23:53 2022] GET /compute/v2.1/servers/05bed8e9-f530-418a-9819-4f65dabdcda0 => generated 1521 bytes in 307 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:23:54.625174 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-bcbc99cd-568a-4e03-ac2e-2836c8a64514 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Calling method '>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:23:54.626001 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-bcbc99cd-568a-4e03-ac2e-2836c8a64514 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] [instance: 05bed8e9-f530-418a-9819-4f65dabdcda0] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:23:54.634769 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-bcbc99cd-568a-4e03-ac2e-2836c8a64514 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:23:54.635416 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-bcbc99cd-568a-4e03-ac2e-2836c8a64514 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:23:54.636050 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-bcbc99cd-568a-4e03-ac2e-2836c8a64514 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:23:54.637274 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-bcbc99cd-568a-4e03-ac2e-2836c8a64514 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:23:54.637955 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-bcbc99cd-568a-4e03-ac2e-2836c8a64514 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:23:54.638648 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-bcbc99cd-568a-4e03-ac2e-2836c8a64514 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:23:54.809527 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-bcbc99cd-568a-4e03-ac2e-2836c8a64514 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd945d512ae034ec08838d0a3afde2429', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '880ac4003248441c809a76cfd1b221eb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:23:54.832842 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-bcbc99cd-568a-4e03-ac2e-2836c8a64514 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] 10.222.0.52 "GET /compute/v2.1/servers/05bed8e9-f530-418a-9819-4f65dabdcda0" status: 200 len: 1521 microversion: 2.1 time: 0.211866 Aug 27 02:23:54.833883 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 107/217] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:23:54 2022] GET /compute/v2.1/servers/05bed8e9-f530-418a-9819-4f65dabdcda0 => generated 1521 bytes in 214 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:23:55.855710 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-c3380a7d-9191-4131-8db1-b8c2b696e53f tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:23:55.856291 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-c3380a7d-9191-4131-8db1-b8c2b696e53f tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] [instance: 05bed8e9-f530-418a-9819-4f65dabdcda0] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:23:55.863094 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-c3380a7d-9191-4131-8db1-b8c2b696e53f tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:23:55.863676 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-c3380a7d-9191-4131-8db1-b8c2b696e53f tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:23:55.864349 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-c3380a7d-9191-4131-8db1-b8c2b696e53f tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:23:55.865247 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-c3380a7d-9191-4131-8db1-b8c2b696e53f tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:23:55.865704 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-c3380a7d-9191-4131-8db1-b8c2b696e53f tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:23:55.866133 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-c3380a7d-9191-4131-8db1-b8c2b696e53f tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:23:56.016311 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-c3380a7d-9191-4131-8db1-b8c2b696e53f tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd945d512ae034ec08838d0a3afde2429', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '880ac4003248441c809a76cfd1b221eb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:23:56.025759 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-c3380a7d-9191-4131-8db1-b8c2b696e53f tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] 10.222.0.52 "GET /compute/v2.1/servers/05bed8e9-f530-418a-9819-4f65dabdcda0" status: 200 len: 1521 microversion: 2.1 time: 0.173809 Aug 27 02:23:56.026349 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 111/218] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:23:55 2022] GET /compute/v2.1/servers/05bed8e9-f530-418a-9819-4f65dabdcda0 => generated 1521 bytes in 175 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:23:57.051428 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-0e616e71-33af-4cc0-a982-35d8c32fdf7a tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Calling method '>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:23:57.052324 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-0e616e71-33af-4cc0-a982-35d8c32fdf7a tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] [instance: 05bed8e9-f530-418a-9819-4f65dabdcda0] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:23:57.059225 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-0e616e71-33af-4cc0-a982-35d8c32fdf7a tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:23:57.059717 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-0e616e71-33af-4cc0-a982-35d8c32fdf7a tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:23:57.060150 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-0e616e71-33af-4cc0-a982-35d8c32fdf7a tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:23:57.060852 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-0e616e71-33af-4cc0-a982-35d8c32fdf7a tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:23:57.061351 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-0e616e71-33af-4cc0-a982-35d8c32fdf7a tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:23:57.061774 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-0e616e71-33af-4cc0-a982-35d8c32fdf7a tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:23:57.204516 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-0e616e71-33af-4cc0-a982-35d8c32fdf7a tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd945d512ae034ec08838d0a3afde2429', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '880ac4003248441c809a76cfd1b221eb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:23:57.212799 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-0e616e71-33af-4cc0-a982-35d8c32fdf7a tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] 10.222.0.52 "GET /compute/v2.1/servers/05bed8e9-f530-418a-9819-4f65dabdcda0" status: 200 len: 1521 microversion: 2.1 time: 0.167817 Aug 27 02:23:57.213400 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 108/219] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:23:57 2022] GET /compute/v2.1/servers/05bed8e9-f530-418a-9819-4f65dabdcda0 => generated 1521 bytes in 170 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:23:58.234409 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-bd28c99a-ad0a-41b2-868b-069f265fafa6 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:23:58.235045 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-bd28c99a-ad0a-41b2-868b-069f265fafa6 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] [instance: 05bed8e9-f530-418a-9819-4f65dabdcda0] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:23:58.241900 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-bd28c99a-ad0a-41b2-868b-069f265fafa6 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:23:58.242385 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-bd28c99a-ad0a-41b2-868b-069f265fafa6 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:23:58.242814 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-bd28c99a-ad0a-41b2-868b-069f265fafa6 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:23:58.243506 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-bd28c99a-ad0a-41b2-868b-069f265fafa6 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:23:58.243954 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-bd28c99a-ad0a-41b2-868b-069f265fafa6 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:23:58.244374 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-bd28c99a-ad0a-41b2-868b-069f265fafa6 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:23:58.390198 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-bd28c99a-ad0a-41b2-868b-069f265fafa6 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd945d512ae034ec08838d0a3afde2429', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '880ac4003248441c809a76cfd1b221eb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:23:58.398702 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-bd28c99a-ad0a-41b2-868b-069f265fafa6 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] 10.222.0.52 "GET /compute/v2.1/servers/05bed8e9-f530-418a-9819-4f65dabdcda0" status: 200 len: 1521 microversion: 2.1 time: 0.168246 Aug 27 02:23:58.399266 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 112/220] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:23:58 2022] GET /compute/v2.1/servers/05bed8e9-f530-418a-9819-4f65dabdcda0 => generated 1521 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:23:59.421168 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-e9f31aa2-8c9a-495c-abc3-157abbd59e9c tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Calling method '>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:23:59.421778 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-e9f31aa2-8c9a-495c-abc3-157abbd59e9c tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] [instance: 05bed8e9-f530-418a-9819-4f65dabdcda0] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:23:59.425777 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG dbcounter [-] [120864] Writing DB stats nova_api:SELECT=60 {{(pid=120864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 27 02:23:59.430564 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-e9f31aa2-8c9a-495c-abc3-157abbd59e9c tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:23:59.431266 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-e9f31aa2-8c9a-495c-abc3-157abbd59e9c tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:23:59.431655 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-e9f31aa2-8c9a-495c-abc3-157abbd59e9c tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:23:59.435645 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-e9f31aa2-8c9a-495c-abc3-157abbd59e9c tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:23:59.436113 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-e9f31aa2-8c9a-495c-abc3-157abbd59e9c tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:23:59.436539 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-e9f31aa2-8c9a-495c-abc3-157abbd59e9c tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:23:59.641916 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-e9f31aa2-8c9a-495c-abc3-157abbd59e9c tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd945d512ae034ec08838d0a3afde2429', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '880ac4003248441c809a76cfd1b221eb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:23:59.650330 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-e9f31aa2-8c9a-495c-abc3-157abbd59e9c tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] 10.222.0.52 "GET /compute/v2.1/servers/05bed8e9-f530-418a-9819-4f65dabdcda0" status: 200 len: 1521 microversion: 2.1 time: 0.234261 Aug 27 02:23:59.663853 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 109/221] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:23:59 2022] GET /compute/v2.1/servers/05bed8e9-f530-418a-9819-4f65dabdcda0 => generated 1521 bytes in 248 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 27 02:23:59.888322 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-6888028d-08a8-41e8-8b83-b574e4b68b5c tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "9a71eb00-42f6-478b-87ec-91c24fb660e4"}} {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 27 02:23:59.891455 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-6888028d-08a8-41e8-8b83-b574e4b68b5c tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] [instance: fbef28bb-3cdf-4a27-8493-add7806af828] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:23:59.898153 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-6888028d-08a8-41e8-8b83-b574e4b68b5c tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:23:59.898628 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-6888028d-08a8-41e8-8b83-b574e4b68b5c tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:23:59.899051 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-6888028d-08a8-41e8-8b83-b574e4b68b5c tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:23:59.899734 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-6888028d-08a8-41e8-8b83-b574e4b68b5c tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:23:59.900175 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-6888028d-08a8-41e8-8b83-b574e4b68b5c tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:23:59.900593 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-6888028d-08a8-41e8-8b83-b574e4b68b5c tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:24:00.672112 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-749b277e-61df-4483-aefb-bb601c38a679 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Calling method '>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:24:00.672790 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-749b277e-61df-4483-aefb-bb601c38a679 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] [instance: 05bed8e9-f530-418a-9819-4f65dabdcda0] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:24:00.681717 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-749b277e-61df-4483-aefb-bb601c38a679 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:24:00.682182 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-749b277e-61df-4483-aefb-bb601c38a679 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:24:00.682880 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-749b277e-61df-4483-aefb-bb601c38a679 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:24:00.683864 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-749b277e-61df-4483-aefb-bb601c38a679 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:24:00.684293 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-749b277e-61df-4483-aefb-bb601c38a679 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:24:00.684697 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-749b277e-61df-4483-aefb-bb601c38a679 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:24:00.705284 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG dbcounter [-] [120864] Writing DB stats nova_cell1:SELECT=100 {{(pid=120864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 27 02:24:00.835792 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-6888028d-08a8-41e8-8b83-b574e4b68b5c tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] 10.222.0.52 "POST /compute/v2.1/servers/fbef28bb-3cdf-4a27-8493-add7806af828/os-volume_attachments" status: 200 len: 194 microversion: 2.1 time: 0.951087 Aug 27 02:24:00.837020 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 113/222] 10.222.0.52 () {60 vars in 1323 bytes} [Sat Aug 27 02:23:59 2022] POST /compute/v2.1/servers/fbef28bb-3cdf-4a27-8493-add7806af828/os-volume_attachments => generated 194 bytes in 954 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 27 02:24:00.855864 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-749b277e-61df-4483-aefb-bb601c38a679 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd945d512ae034ec08838d0a3afde2429', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '880ac4003248441c809a76cfd1b221eb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:24:00.870988 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-749b277e-61df-4483-aefb-bb601c38a679 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] 10.222.0.52 "GET /compute/v2.1/servers/05bed8e9-f530-418a-9819-4f65dabdcda0" status: 200 len: 1521 microversion: 2.1 time: 0.204046 Aug 27 02:24:00.872903 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 110/223] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:24:00 2022] GET /compute/v2.1/servers/05bed8e9-f530-418a-9819-4f65dabdcda0 => generated 1521 bytes in 206 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 27 02:24:01.894339 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-d341ce85-0a7d-4640-9da0-7318ce55d427 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:24:01.895558 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-d341ce85-0a7d-4640-9da0-7318ce55d427 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] [instance: 05bed8e9-f530-418a-9819-4f65dabdcda0] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:24:01.903646 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-d341ce85-0a7d-4640-9da0-7318ce55d427 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:24:01.904321 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-d341ce85-0a7d-4640-9da0-7318ce55d427 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:24:01.904794 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-d341ce85-0a7d-4640-9da0-7318ce55d427 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:24:01.905629 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-d341ce85-0a7d-4640-9da0-7318ce55d427 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:24:01.906075 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-d341ce85-0a7d-4640-9da0-7318ce55d427 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:24:01.906492 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-d341ce85-0a7d-4640-9da0-7318ce55d427 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:24:02.078936 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-d341ce85-0a7d-4640-9da0-7318ce55d427 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd945d512ae034ec08838d0a3afde2429', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '880ac4003248441c809a76cfd1b221eb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:24:02.088662 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-d341ce85-0a7d-4640-9da0-7318ce55d427 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] 10.222.0.52 "GET /compute/v2.1/servers/05bed8e9-f530-418a-9819-4f65dabdcda0" status: 200 len: 1521 microversion: 2.1 time: 0.198890 Aug 27 02:24:02.089648 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 114/224] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:24:01 2022] GET /compute/v2.1/servers/05bed8e9-f530-418a-9819-4f65dabdcda0 => generated 1521 bytes in 200 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:24:03.114800 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-ec4c4211-c1d1-4c4e-a319-87733a48b3fd tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Calling method '>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:24:03.115684 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-ec4c4211-c1d1-4c4e-a319-87733a48b3fd tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] [instance: 05bed8e9-f530-418a-9819-4f65dabdcda0] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:24:03.124594 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-ec4c4211-c1d1-4c4e-a319-87733a48b3fd tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:24:03.126259 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-ec4c4211-c1d1-4c4e-a319-87733a48b3fd tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:24:03.126861 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-ec4c4211-c1d1-4c4e-a319-87733a48b3fd tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:24:03.128149 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-ec4c4211-c1d1-4c4e-a319-87733a48b3fd tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:24:03.129215 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-ec4c4211-c1d1-4c4e-a319-87733a48b3fd tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:24:03.129805 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-ec4c4211-c1d1-4c4e-a319-87733a48b3fd tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:24:03.298811 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-ec4c4211-c1d1-4c4e-a319-87733a48b3fd tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd945d512ae034ec08838d0a3afde2429', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '880ac4003248441c809a76cfd1b221eb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:24:03.308196 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-ec4c4211-c1d1-4c4e-a319-87733a48b3fd tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] 10.222.0.52 "GET /compute/v2.1/servers/05bed8e9-f530-418a-9819-4f65dabdcda0" status: 200 len: 1521 microversion: 2.1 time: 0.199569 Aug 27 02:24:03.308756 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 111/225] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:24:03 2022] GET /compute/v2.1/servers/05bed8e9-f530-418a-9819-4f65dabdcda0 => generated 1521 bytes in 201 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:24:04.332172 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-355b0b18-3cf0-4fd0-bf65-858835f3e7f2 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:24:04.333567 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-355b0b18-3cf0-4fd0-bf65-858835f3e7f2 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] [instance: 05bed8e9-f530-418a-9819-4f65dabdcda0] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:24:04.341053 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-355b0b18-3cf0-4fd0-bf65-858835f3e7f2 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:24:04.341613 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-355b0b18-3cf0-4fd0-bf65-858835f3e7f2 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:24:04.342154 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-355b0b18-3cf0-4fd0-bf65-858835f3e7f2 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:24:04.342900 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-355b0b18-3cf0-4fd0-bf65-858835f3e7f2 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:24:04.343438 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-355b0b18-3cf0-4fd0-bf65-858835f3e7f2 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:24:04.343915 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-355b0b18-3cf0-4fd0-bf65-858835f3e7f2 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:24:04.515180 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-355b0b18-3cf0-4fd0-bf65-858835f3e7f2 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd945d512ae034ec08838d0a3afde2429', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '880ac4003248441c809a76cfd1b221eb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:24:04.523705 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-355b0b18-3cf0-4fd0-bf65-858835f3e7f2 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] 10.222.0.52 "GET /compute/v2.1/servers/05bed8e9-f530-418a-9819-4f65dabdcda0" status: 200 len: 1521 microversion: 2.1 time: 0.197395 Aug 27 02:24:04.524721 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 115/226] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:24:04 2022] GET /compute/v2.1/servers/05bed8e9-f530-418a-9819-4f65dabdcda0 => generated 1521 bytes in 199 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:24:05.544867 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-01b90f01-9132-489a-84e9-2886e09142ba tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Calling method '>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:24:05.545519 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-01b90f01-9132-489a-84e9-2886e09142ba tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] [instance: 05bed8e9-f530-418a-9819-4f65dabdcda0] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:24:05.551982 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-01b90f01-9132-489a-84e9-2886e09142ba tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:24:05.552468 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-01b90f01-9132-489a-84e9-2886e09142ba tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:24:05.552941 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-01b90f01-9132-489a-84e9-2886e09142ba tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:24:05.553668 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-01b90f01-9132-489a-84e9-2886e09142ba tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:24:05.554307 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-01b90f01-9132-489a-84e9-2886e09142ba tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:24:05.554724 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-01b90f01-9132-489a-84e9-2886e09142ba tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:24:05.692321 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-01b90f01-9132-489a-84e9-2886e09142ba tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd945d512ae034ec08838d0a3afde2429', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '880ac4003248441c809a76cfd1b221eb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:24:05.700761 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-01b90f01-9132-489a-84e9-2886e09142ba tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] 10.222.0.52 "GET /compute/v2.1/servers/05bed8e9-f530-418a-9819-4f65dabdcda0" status: 200 len: 1521 microversion: 2.1 time: 0.159818 Aug 27 02:24:05.701455 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 112/227] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:24:05 2022] GET /compute/v2.1/servers/05bed8e9-f530-418a-9819-4f65dabdcda0 => generated 1521 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:24:06.729717 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-9b31a736-3b44-47dd-9049-9cbb25a7d6c3 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:24:06.730360 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-9b31a736-3b44-47dd-9049-9cbb25a7d6c3 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] [instance: 05bed8e9-f530-418a-9819-4f65dabdcda0] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:24:06.745658 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-9b31a736-3b44-47dd-9049-9cbb25a7d6c3 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:24:06.746165 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-9b31a736-3b44-47dd-9049-9cbb25a7d6c3 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:24:06.746675 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-9b31a736-3b44-47dd-9049-9cbb25a7d6c3 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:24:06.747485 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-9b31a736-3b44-47dd-9049-9cbb25a7d6c3 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:24:06.748017 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-9b31a736-3b44-47dd-9049-9cbb25a7d6c3 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:24:06.749405 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-9b31a736-3b44-47dd-9049-9cbb25a7d6c3 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:24:07.111913 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-9b31a736-3b44-47dd-9049-9cbb25a7d6c3 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd945d512ae034ec08838d0a3afde2429', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '880ac4003248441c809a76cfd1b221eb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:24:07.121320 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-9b31a736-3b44-47dd-9049-9cbb25a7d6c3 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] 10.222.0.52 "GET /compute/v2.1/servers/05bed8e9-f530-418a-9819-4f65dabdcda0" status: 200 len: 1521 microversion: 2.1 time: 0.401254 Aug 27 02:24:07.122099 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 116/228] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:24:06 2022] GET /compute/v2.1/servers/05bed8e9-f530-418a-9819-4f65dabdcda0 => generated 1521 bytes in 403 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:24:08.144042 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-9a3a1f62-34d2-49c9-8a51-2fe72f3ed9f0 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Calling method '>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:24:08.145176 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-9a3a1f62-34d2-49c9-8a51-2fe72f3ed9f0 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] [instance: 05bed8e9-f530-418a-9819-4f65dabdcda0] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:24:08.152065 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-9a3a1f62-34d2-49c9-8a51-2fe72f3ed9f0 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:24:08.152731 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-9a3a1f62-34d2-49c9-8a51-2fe72f3ed9f0 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:24:08.153413 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-9a3a1f62-34d2-49c9-8a51-2fe72f3ed9f0 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:24:08.154309 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-9a3a1f62-34d2-49c9-8a51-2fe72f3ed9f0 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:24:08.155174 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-9a3a1f62-34d2-49c9-8a51-2fe72f3ed9f0 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:24:08.155963 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-9a3a1f62-34d2-49c9-8a51-2fe72f3ed9f0 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:24:08.362301 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-9a3a1f62-34d2-49c9-8a51-2fe72f3ed9f0 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd945d512ae034ec08838d0a3afde2429', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '880ac4003248441c809a76cfd1b221eb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:24:08.370798 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-9a3a1f62-34d2-49c9-8a51-2fe72f3ed9f0 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] 10.222.0.52 "GET /compute/v2.1/servers/05bed8e9-f530-418a-9819-4f65dabdcda0" status: 200 len: 1521 microversion: 2.1 time: 0.230871 Aug 27 02:24:08.371510 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 113/229] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:24:08 2022] GET /compute/v2.1/servers/05bed8e9-f530-418a-9819-4f65dabdcda0 => generated 1521 bytes in 232 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:24:09.393006 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-fdbd3674-dcde-429a-be97-9f3e92e434b7 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:24:09.393678 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-fdbd3674-dcde-429a-be97-9f3e92e434b7 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] [instance: 05bed8e9-f530-418a-9819-4f65dabdcda0] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:24:09.400406 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-fdbd3674-dcde-429a-be97-9f3e92e434b7 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:24:09.401012 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-fdbd3674-dcde-429a-be97-9f3e92e434b7 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:24:09.401498 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-fdbd3674-dcde-429a-be97-9f3e92e434b7 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:24:09.402205 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-fdbd3674-dcde-429a-be97-9f3e92e434b7 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:24:09.402650 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-fdbd3674-dcde-429a-be97-9f3e92e434b7 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:24:09.403158 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-fdbd3674-dcde-429a-be97-9f3e92e434b7 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:24:09.424242 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG dbcounter [-] [120863] Writing DB stats nova_cell1:SELECT=94,nova_cell1:UPDATE=4,nova_cell1:INSERT=2 {{(pid=120863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 27 02:24:09.554845 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-fdbd3674-dcde-429a-be97-9f3e92e434b7 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd945d512ae034ec08838d0a3afde2429', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '880ac4003248441c809a76cfd1b221eb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:24:09.562668 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-fdbd3674-dcde-429a-be97-9f3e92e434b7 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] 10.222.0.52 "GET /compute/v2.1/servers/05bed8e9-f530-418a-9819-4f65dabdcda0" status: 200 len: 1521 microversion: 2.1 time: 0.174167 Aug 27 02:24:09.563579 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 117/230] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:24:09 2022] GET /compute/v2.1/servers/05bed8e9-f530-418a-9819-4f65dabdcda0 => generated 1521 bytes in 176 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:24:10.585225 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-cc21032f-97da-4a8d-bfdf-1cc966f742ef tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Calling method '>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:24:10.586088 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-cc21032f-97da-4a8d-bfdf-1cc966f742ef tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] [instance: 05bed8e9-f530-418a-9819-4f65dabdcda0] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:24:10.594734 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-cc21032f-97da-4a8d-bfdf-1cc966f742ef tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:24:10.595334 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-cc21032f-97da-4a8d-bfdf-1cc966f742ef tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:24:10.596401 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-cc21032f-97da-4a8d-bfdf-1cc966f742ef tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:24:10.597440 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-cc21032f-97da-4a8d-bfdf-1cc966f742ef tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:24:10.598206 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-cc21032f-97da-4a8d-bfdf-1cc966f742ef tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:24:10.598833 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-cc21032f-97da-4a8d-bfdf-1cc966f742ef tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:24:10.639861 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-e2d502b7-1831-4b38-81e7-7d76697883be tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:24:10.640435 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-e2d502b7-1831-4b38-81e7-7d76697883be tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] [instance: fbef28bb-3cdf-4a27-8493-add7806af828] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:24:10.647517 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-e2d502b7-1831-4b38-81e7-7d76697883be tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:24:10.648039 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-e2d502b7-1831-4b38-81e7-7d76697883be tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:24:10.648699 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-e2d502b7-1831-4b38-81e7-7d76697883be tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:24:10.649544 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-e2d502b7-1831-4b38-81e7-7d76697883be tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:24:10.650149 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-e2d502b7-1831-4b38-81e7-7d76697883be tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:24:10.650750 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-e2d502b7-1831-4b38-81e7-7d76697883be tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:24:10.679236 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-e2d502b7-1831-4b38-81e7-7d76697883be tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] [instance: fbef28bb-3cdf-4a27-8493-add7806af828] Going to try to terminate instance {{(pid=120863) delete /opt/stack/nova/nova/compute/api.py:2719}} Aug 27 02:24:10.775286 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-cc21032f-97da-4a8d-bfdf-1cc966f742ef tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd945d512ae034ec08838d0a3afde2429', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '880ac4003248441c809a76cfd1b221eb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:24:10.786565 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-cc21032f-97da-4a8d-bfdf-1cc966f742ef tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] 10.222.0.52 "GET /compute/v2.1/servers/05bed8e9-f530-418a-9819-4f65dabdcda0" status: 200 len: 1521 microversion: 2.1 time: 0.205699 Aug 27 02:24:10.787681 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 114/231] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:24:10 2022] GET /compute/v2.1/servers/05bed8e9-f530-418a-9819-4f65dabdcda0 => generated 1521 bytes in 208 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:24:10.792096 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-e2d502b7-1831-4b38-81e7-7d76697883be tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] 10.222.0.52 "DELETE /compute/v2.1/servers/fbef28bb-3cdf-4a27-8493-add7806af828" status: 204 len: 0 microversion: 2.1 time: 0.155910 Aug 27 02:24:10.792699 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 118/232] 10.222.0.52 () {58 vars in 1239 bytes} [Sat Aug 27 02:24:10 2022] DELETE /compute/v2.1/servers/fbef28bb-3cdf-4a27-8493-add7806af828 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 9 headers in 363 bytes (2 switches on core 0) Aug 27 02:24:10.806448 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-db83313b-4226-40de-b250-3f1f46c67266 tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Calling method '>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:24:10.807139 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-db83313b-4226-40de-b250-3f1f46c67266 tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] [instance: fbef28bb-3cdf-4a27-8493-add7806af828] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:24:10.813559 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-db83313b-4226-40de-b250-3f1f46c67266 tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:24:10.814052 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-db83313b-4226-40de-b250-3f1f46c67266 tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:24:10.814485 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-db83313b-4226-40de-b250-3f1f46c67266 tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:24:10.815177 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-db83313b-4226-40de-b250-3f1f46c67266 tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:24:10.815624 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-db83313b-4226-40de-b250-3f1f46c67266 tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:24:10.816046 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-db83313b-4226-40de-b250-3f1f46c67266 tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:24:10.958171 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-db83313b-4226-40de-b250-3f1f46c67266 tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '55640f750d1b49be9c3f79501a63c1ab', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7ef4b44a3f94d599eacce269b06594c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:24:10.967397 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-db83313b-4226-40de-b250-3f1f46c67266 tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] 10.222.0.52 "GET /compute/v2.1/servers/fbef28bb-3cdf-4a27-8493-add7806af828" status: 200 len: 1554 microversion: 2.1 time: 0.168752 Aug 27 02:24:10.967925 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 115/233] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:24:10 2022] GET /compute/v2.1/servers/fbef28bb-3cdf-4a27-8493-add7806af828 => generated 1554 bytes in 170 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 27 02:24:11.810648 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-21485df6-a6ed-4d24-8a16-a93289320127 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:24:11.811406 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-21485df6-a6ed-4d24-8a16-a93289320127 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] [instance: 05bed8e9-f530-418a-9819-4f65dabdcda0] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:24:11.818030 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-21485df6-a6ed-4d24-8a16-a93289320127 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:24:11.818495 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-21485df6-a6ed-4d24-8a16-a93289320127 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:24:11.818926 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-21485df6-a6ed-4d24-8a16-a93289320127 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:24:11.819619 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-21485df6-a6ed-4d24-8a16-a93289320127 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:24:11.820066 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-21485df6-a6ed-4d24-8a16-a93289320127 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:24:11.820614 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-21485df6-a6ed-4d24-8a16-a93289320127 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:24:11.958679 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-21485df6-a6ed-4d24-8a16-a93289320127 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd945d512ae034ec08838d0a3afde2429', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '880ac4003248441c809a76cfd1b221eb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:24:11.967997 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-21485df6-a6ed-4d24-8a16-a93289320127 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] 10.222.0.52 "GET /compute/v2.1/servers/05bed8e9-f530-418a-9819-4f65dabdcda0" status: 200 len: 1521 microversion: 2.1 time: 0.161640 Aug 27 02:24:11.968576 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 119/234] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:24:11 2022] GET /compute/v2.1/servers/05bed8e9-f530-418a-9819-4f65dabdcda0 => generated 1521 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:24:11.988702 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-17963760-331e-41f3-863b-401474bc6f0d tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Calling method '>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:24:11.989412 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-17963760-331e-41f3-863b-401474bc6f0d tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] [instance: fbef28bb-3cdf-4a27-8493-add7806af828] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:24:11.996264 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-17963760-331e-41f3-863b-401474bc6f0d tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:24:11.996264 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-17963760-331e-41f3-863b-401474bc6f0d tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:24:11.996690 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-17963760-331e-41f3-863b-401474bc6f0d tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:24:11.997598 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-17963760-331e-41f3-863b-401474bc6f0d tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:24:11.999264 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-17963760-331e-41f3-863b-401474bc6f0d tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:24:11.999264 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-17963760-331e-41f3-863b-401474bc6f0d tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:24:12.134756 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-17963760-331e-41f3-863b-401474bc6f0d tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '55640f750d1b49be9c3f79501a63c1ab', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7ef4b44a3f94d599eacce269b06594c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:24:12.143076 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-17963760-331e-41f3-863b-401474bc6f0d tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] 10.222.0.52 "GET /compute/v2.1/servers/fbef28bb-3cdf-4a27-8493-add7806af828" status: 200 len: 1554 microversion: 2.1 time: 0.158438 Aug 27 02:24:12.143663 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 116/235] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:24:11 2022] GET /compute/v2.1/servers/fbef28bb-3cdf-4a27-8493-add7806af828 => generated 1554 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:24:12.989873 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-57e2a504-4763-4caf-91f7-d0ef7d45fe0c tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:24:12.990790 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-57e2a504-4763-4caf-91f7-d0ef7d45fe0c tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] [instance: 05bed8e9-f530-418a-9819-4f65dabdcda0] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:24:12.997882 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-57e2a504-4763-4caf-91f7-d0ef7d45fe0c tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:24:12.998716 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-57e2a504-4763-4caf-91f7-d0ef7d45fe0c tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:24:12.999281 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-57e2a504-4763-4caf-91f7-d0ef7d45fe0c tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:24:13.000394 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-57e2a504-4763-4caf-91f7-d0ef7d45fe0c tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:24:13.001104 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-57e2a504-4763-4caf-91f7-d0ef7d45fe0c tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:24:13.001902 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-57e2a504-4763-4caf-91f7-d0ef7d45fe0c tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:24:13.139367 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-57e2a504-4763-4caf-91f7-d0ef7d45fe0c tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd945d512ae034ec08838d0a3afde2429', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '880ac4003248441c809a76cfd1b221eb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:24:13.147552 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-57e2a504-4763-4caf-91f7-d0ef7d45fe0c tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] 10.222.0.52 "GET /compute/v2.1/servers/05bed8e9-f530-418a-9819-4f65dabdcda0" status: 200 len: 1521 microversion: 2.1 time: 0.162263 Aug 27 02:24:13.148117 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 120/236] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:24:12 2022] GET /compute/v2.1/servers/05bed8e9-f530-418a-9819-4f65dabdcda0 => generated 1521 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:24:13.164237 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-9aebba00-4dda-43ca-bd75-6158067d3696 tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Calling method '>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:24:13.164837 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-9aebba00-4dda-43ca-bd75-6158067d3696 tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] [instance: fbef28bb-3cdf-4a27-8493-add7806af828] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:24:13.171370 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-9aebba00-4dda-43ca-bd75-6158067d3696 tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:24:13.171946 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-9aebba00-4dda-43ca-bd75-6158067d3696 tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:24:13.172376 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-9aebba00-4dda-43ca-bd75-6158067d3696 tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:24:13.173148 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-9aebba00-4dda-43ca-bd75-6158067d3696 tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:24:13.173586 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-9aebba00-4dda-43ca-bd75-6158067d3696 tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:24:13.174004 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-9aebba00-4dda-43ca-bd75-6158067d3696 tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:24:13.317799 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-9aebba00-4dda-43ca-bd75-6158067d3696 tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '55640f750d1b49be9c3f79501a63c1ab', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7ef4b44a3f94d599eacce269b06594c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:24:13.326472 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-9aebba00-4dda-43ca-bd75-6158067d3696 tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] 10.222.0.52 "GET /compute/v2.1/servers/fbef28bb-3cdf-4a27-8493-add7806af828" status: 200 len: 1554 microversion: 2.1 time: 0.166193 Aug 27 02:24:13.327071 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 117/237] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:24:13 2022] GET /compute/v2.1/servers/fbef28bb-3cdf-4a27-8493-add7806af828 => generated 1554 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:24:14.168176 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-a6ace83a-096d-4837-8da6-addc807b0b1d tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:24:14.168778 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-a6ace83a-096d-4837-8da6-addc807b0b1d tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] [instance: 05bed8e9-f530-418a-9819-4f65dabdcda0] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:24:14.176059 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-a6ace83a-096d-4837-8da6-addc807b0b1d tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:24:14.176544 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-a6ace83a-096d-4837-8da6-addc807b0b1d tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:24:14.176970 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-a6ace83a-096d-4837-8da6-addc807b0b1d tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:24:14.177699 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-a6ace83a-096d-4837-8da6-addc807b0b1d tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:24:14.178150 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-a6ace83a-096d-4837-8da6-addc807b0b1d tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:24:14.178572 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-a6ace83a-096d-4837-8da6-addc807b0b1d tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:24:14.330958 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-a6ace83a-096d-4837-8da6-addc807b0b1d tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd945d512ae034ec08838d0a3afde2429', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '880ac4003248441c809a76cfd1b221eb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:24:14.340044 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-a6ace83a-096d-4837-8da6-addc807b0b1d tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] 10.222.0.52 "GET /compute/v2.1/servers/05bed8e9-f530-418a-9819-4f65dabdcda0" status: 200 len: 1521 microversion: 2.1 time: 0.175734 Aug 27 02:24:14.340629 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 121/238] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:24:14 2022] GET /compute/v2.1/servers/05bed8e9-f530-418a-9819-4f65dabdcda0 => generated 1521 bytes in 177 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:24:14.347261 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-c1451371-ba08-45dd-8101-c4a29abf9e49 tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Calling method '>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:24:14.348259 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-c1451371-ba08-45dd-8101-c4a29abf9e49 tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] [instance: fbef28bb-3cdf-4a27-8493-add7806af828] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:24:14.355784 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-c1451371-ba08-45dd-8101-c4a29abf9e49 tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:24:14.356263 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-c1451371-ba08-45dd-8101-c4a29abf9e49 tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:24:14.356691 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-c1451371-ba08-45dd-8101-c4a29abf9e49 tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:24:14.357426 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-c1451371-ba08-45dd-8101-c4a29abf9e49 tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:24:14.357871 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-c1451371-ba08-45dd-8101-c4a29abf9e49 tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:24:14.358288 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-c1451371-ba08-45dd-8101-c4a29abf9e49 tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:24:14.503293 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-c1451371-ba08-45dd-8101-c4a29abf9e49 tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '55640f750d1b49be9c3f79501a63c1ab', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7ef4b44a3f94d599eacce269b06594c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:24:14.514381 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-c1451371-ba08-45dd-8101-c4a29abf9e49 tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] 10.222.0.52 "GET /compute/v2.1/servers/fbef28bb-3cdf-4a27-8493-add7806af828" status: 200 len: 1554 microversion: 2.1 time: 0.170963 Aug 27 02:24:14.515086 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 118/239] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:24:14 2022] GET /compute/v2.1/servers/fbef28bb-3cdf-4a27-8493-add7806af828 => generated 1554 bytes in 172 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:24:15.366879 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-12ab806b-0002-47ce-a201-1c7303c1cb61 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:24:15.367686 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-12ab806b-0002-47ce-a201-1c7303c1cb61 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] [instance: 05bed8e9-f530-418a-9819-4f65dabdcda0] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:24:15.376217 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-12ab806b-0002-47ce-a201-1c7303c1cb61 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:24:15.376783 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-12ab806b-0002-47ce-a201-1c7303c1cb61 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:24:15.377365 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-12ab806b-0002-47ce-a201-1c7303c1cb61 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:24:15.378213 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-12ab806b-0002-47ce-a201-1c7303c1cb61 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:24:15.379145 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-12ab806b-0002-47ce-a201-1c7303c1cb61 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:24:15.380955 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-12ab806b-0002-47ce-a201-1c7303c1cb61 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:24:15.517360 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-12ab806b-0002-47ce-a201-1c7303c1cb61 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd945d512ae034ec08838d0a3afde2429', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '880ac4003248441c809a76cfd1b221eb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:24:15.526298 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-12ab806b-0002-47ce-a201-1c7303c1cb61 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] 10.222.0.52 "GET /compute/v2.1/servers/05bed8e9-f530-418a-9819-4f65dabdcda0" status: 200 len: 1521 microversion: 2.1 time: 0.164577 Aug 27 02:24:15.526874 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 122/240] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:24:15 2022] GET /compute/v2.1/servers/05bed8e9-f530-418a-9819-4f65dabdcda0 => generated 1521 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 27 02:24:15.535837 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-971db8eb-7a12-44a2-9341-a412d0c04cd0 tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Calling method '>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:24:15.536444 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-971db8eb-7a12-44a2-9341-a412d0c04cd0 tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] [instance: fbef28bb-3cdf-4a27-8493-add7806af828] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:24:15.542785 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-971db8eb-7a12-44a2-9341-a412d0c04cd0 tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:24:15.543257 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-971db8eb-7a12-44a2-9341-a412d0c04cd0 tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:24:15.543719 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-971db8eb-7a12-44a2-9341-a412d0c04cd0 tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:24:15.544590 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-971db8eb-7a12-44a2-9341-a412d0c04cd0 tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:24:15.545074 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-971db8eb-7a12-44a2-9341-a412d0c04cd0 tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:24:15.545547 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-971db8eb-7a12-44a2-9341-a412d0c04cd0 tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:24:15.686324 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-971db8eb-7a12-44a2-9341-a412d0c04cd0 tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '55640f750d1b49be9c3f79501a63c1ab', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7ef4b44a3f94d599eacce269b06594c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:24:15.694783 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-971db8eb-7a12-44a2-9341-a412d0c04cd0 tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] 10.222.0.52 "GET /compute/v2.1/servers/fbef28bb-3cdf-4a27-8493-add7806af828" status: 200 len: 1554 microversion: 2.1 time: 0.162878 Aug 27 02:24:15.695338 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 119/241] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:24:15 2022] GET /compute/v2.1/servers/fbef28bb-3cdf-4a27-8493-add7806af828 => generated 1554 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:24:16.552499 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-c334bbd5-a414-47aa-9dca-ae66018b4f9a tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:24:16.553073 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-c334bbd5-a414-47aa-9dca-ae66018b4f9a tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] [instance: 05bed8e9-f530-418a-9819-4f65dabdcda0] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:24:16.559839 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-c334bbd5-a414-47aa-9dca-ae66018b4f9a tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:24:16.560333 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-c334bbd5-a414-47aa-9dca-ae66018b4f9a tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:24:16.560769 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-c334bbd5-a414-47aa-9dca-ae66018b4f9a tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:24:16.561544 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-c334bbd5-a414-47aa-9dca-ae66018b4f9a tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:24:16.562004 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-c334bbd5-a414-47aa-9dca-ae66018b4f9a tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:24:16.562426 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-c334bbd5-a414-47aa-9dca-ae66018b4f9a tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:24:16.712560 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-c334bbd5-a414-47aa-9dca-ae66018b4f9a tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd945d512ae034ec08838d0a3afde2429', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '880ac4003248441c809a76cfd1b221eb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:24:16.720668 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-9ec50fe0-75ba-4d5d-a63b-81904c5b5680 tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Calling method '>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:24:16.721320 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-9ec50fe0-75ba-4d5d-a63b-81904c5b5680 tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] [instance: fbef28bb-3cdf-4a27-8493-add7806af828] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:24:16.727332 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-c334bbd5-a414-47aa-9dca-ae66018b4f9a tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] 10.222.0.52 "GET /compute/v2.1/servers/05bed8e9-f530-418a-9819-4f65dabdcda0" status: 200 len: 1521 microversion: 2.1 time: 0.180893 Aug 27 02:24:16.727897 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 123/242] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:24:16 2022] GET /compute/v2.1/servers/05bed8e9-f530-418a-9819-4f65dabdcda0 => generated 1521 bytes in 182 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:24:16.728023 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-9ec50fe0-75ba-4d5d-a63b-81904c5b5680 tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:24:16.728457 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-9ec50fe0-75ba-4d5d-a63b-81904c5b5680 tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:24:16.728917 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-9ec50fe0-75ba-4d5d-a63b-81904c5b5680 tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:24:16.729630 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-9ec50fe0-75ba-4d5d-a63b-81904c5b5680 tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:24:16.730075 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-9ec50fe0-75ba-4d5d-a63b-81904c5b5680 tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:24:16.730491 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-9ec50fe0-75ba-4d5d-a63b-81904c5b5680 tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:24:16.867982 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-9ec50fe0-75ba-4d5d-a63b-81904c5b5680 tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '55640f750d1b49be9c3f79501a63c1ab', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7ef4b44a3f94d599eacce269b06594c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:24:16.876341 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-9ec50fe0-75ba-4d5d-a63b-81904c5b5680 tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] 10.222.0.52 "GET /compute/v2.1/servers/fbef28bb-3cdf-4a27-8493-add7806af828" status: 200 len: 1554 microversion: 2.1 time: 0.159543 Aug 27 02:24:16.876938 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 120/243] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:24:16 2022] GET /compute/v2.1/servers/fbef28bb-3cdf-4a27-8493-add7806af828 => generated 1554 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 27 02:24:17.748019 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-dff87e0b-4877-4296-aa96-e95430bd2594 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:24:17.748692 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-dff87e0b-4877-4296-aa96-e95430bd2594 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] [instance: 05bed8e9-f530-418a-9819-4f65dabdcda0] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:24:17.756510 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-dff87e0b-4877-4296-aa96-e95430bd2594 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:24:17.756980 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-dff87e0b-4877-4296-aa96-e95430bd2594 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:24:17.757443 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-dff87e0b-4877-4296-aa96-e95430bd2594 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:24:17.758141 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-dff87e0b-4877-4296-aa96-e95430bd2594 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:24:17.758590 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-dff87e0b-4877-4296-aa96-e95430bd2594 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:24:17.759006 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-dff87e0b-4877-4296-aa96-e95430bd2594 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:24:17.897952 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-6ed0a256-1e31-493d-ae31-79681d439c70 tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Calling method '>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:24:17.898551 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-6ed0a256-1e31-493d-ae31-79681d439c70 tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] [instance: fbef28bb-3cdf-4a27-8493-add7806af828] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:24:17.901890 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-dff87e0b-4877-4296-aa96-e95430bd2594 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd945d512ae034ec08838d0a3afde2429', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '880ac4003248441c809a76cfd1b221eb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:24:17.905112 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-6ed0a256-1e31-493d-ae31-79681d439c70 tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:24:17.905773 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-6ed0a256-1e31-493d-ae31-79681d439c70 tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:24:17.906511 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-6ed0a256-1e31-493d-ae31-79681d439c70 tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:24:17.907359 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-6ed0a256-1e31-493d-ae31-79681d439c70 tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:24:17.907811 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-6ed0a256-1e31-493d-ae31-79681d439c70 tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:24:17.908225 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-6ed0a256-1e31-493d-ae31-79681d439c70 tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:24:17.911615 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-dff87e0b-4877-4296-aa96-e95430bd2594 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] 10.222.0.52 "GET /compute/v2.1/servers/05bed8e9-f530-418a-9819-4f65dabdcda0" status: 200 len: 1521 microversion: 2.1 time: 0.167451 Aug 27 02:24:17.912385 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 124/244] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:24:17 2022] GET /compute/v2.1/servers/05bed8e9-f530-418a-9819-4f65dabdcda0 => generated 1521 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:24:18.051332 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-6ed0a256-1e31-493d-ae31-79681d439c70 tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '55640f750d1b49be9c3f79501a63c1ab', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7ef4b44a3f94d599eacce269b06594c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:24:18.060172 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-6ed0a256-1e31-493d-ae31-79681d439c70 tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] 10.222.0.52 "GET /compute/v2.1/servers/fbef28bb-3cdf-4a27-8493-add7806af828" status: 200 len: 1554 microversion: 2.1 time: 0.166206 Aug 27 02:24:18.060743 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 121/245] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:24:17 2022] GET /compute/v2.1/servers/fbef28bb-3cdf-4a27-8493-add7806af828 => generated 1554 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 27 02:24:18.940005 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-57676b03-6b13-4f3d-b670-219cb58c2ee5 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:24:18.940649 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-57676b03-6b13-4f3d-b670-219cb58c2ee5 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] [instance: 05bed8e9-f530-418a-9819-4f65dabdcda0] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:24:18.947240 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-57676b03-6b13-4f3d-b670-219cb58c2ee5 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:24:18.947723 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-57676b03-6b13-4f3d-b670-219cb58c2ee5 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:24:18.948660 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-57676b03-6b13-4f3d-b670-219cb58c2ee5 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:24:18.949820 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-57676b03-6b13-4f3d-b670-219cb58c2ee5 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:24:18.950269 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-57676b03-6b13-4f3d-b670-219cb58c2ee5 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:24:18.950684 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-57676b03-6b13-4f3d-b670-219cb58c2ee5 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:24:19.083207 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-b000db4d-0066-431d-aa03-3fc72b53072c tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Calling method '>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:24:19.083963 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-b000db4d-0066-431d-aa03-3fc72b53072c tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] [instance: fbef28bb-3cdf-4a27-8493-add7806af828] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:24:19.090664 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-b000db4d-0066-431d-aa03-3fc72b53072c tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:24:19.091561 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-b000db4d-0066-431d-aa03-3fc72b53072c tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:24:19.092235 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-b000db4d-0066-431d-aa03-3fc72b53072c tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:24:19.093160 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-b000db4d-0066-431d-aa03-3fc72b53072c tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:24:19.094091 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-b000db4d-0066-431d-aa03-3fc72b53072c tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:24:19.094716 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-b000db4d-0066-431d-aa03-3fc72b53072c tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:24:19.296059 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-57676b03-6b13-4f3d-b670-219cb58c2ee5 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd945d512ae034ec08838d0a3afde2429', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '880ac4003248441c809a76cfd1b221eb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:24:19.308449 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-57676b03-6b13-4f3d-b670-219cb58c2ee5 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] 10.222.0.52 "GET /compute/v2.1/servers/05bed8e9-f530-418a-9819-4f65dabdcda0" status: 200 len: 1521 microversion: 2.1 time: 0.377768 Aug 27 02:24:19.323552 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 125/246] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:24:18 2022] GET /compute/v2.1/servers/05bed8e9-f530-418a-9819-4f65dabdcda0 => generated 1521 bytes in 395 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:24:19.367460 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-b000db4d-0066-431d-aa03-3fc72b53072c tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '55640f750d1b49be9c3f79501a63c1ab', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7ef4b44a3f94d599eacce269b06594c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:24:19.376807 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-b000db4d-0066-431d-aa03-3fc72b53072c tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] 10.222.0.52 "GET /compute/v2.1/servers/fbef28bb-3cdf-4a27-8493-add7806af828" status: 200 len: 1554 microversion: 2.1 time: 0.297518 Aug 27 02:24:19.377716 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 122/247] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:24:19 2022] GET /compute/v2.1/servers/fbef28bb-3cdf-4a27-8493-add7806af828 => generated 1554 bytes in 299 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:24:19.657828 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [req-6afbc374-07cd-4458-91ba-d6d66ac97f7f req-c3ed39a7-ace0-44d5-a4a5-6f20940d5997 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "fbef28bb-3cdf-4a27-8493-add7806af828", "name": "network-vif-deleted", "tag": "b4168fa6-3b0a-4107-83cf-7d31f29910d4"}]} {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 27 02:24:19.669631 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [req-6afbc374-07cd-4458-91ba-d6d66ac97f7f req-c3ed39a7-ace0-44d5-a4a5-6f20940d5997 service nova] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:24:19.670224 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [req-6afbc374-07cd-4458-91ba-d6d66ac97f7f req-c3ed39a7-ace0-44d5-a4a5-6f20940d5997 service nova] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:24:19.670729 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [req-6afbc374-07cd-4458-91ba-d6d66ac97f7f req-c3ed39a7-ace0-44d5-a4a5-6f20940d5997 service nova] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:24:19.683287 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.compute.server_external_events [req-6afbc374-07cd-4458-91ba-d6d66ac97f7f req-c3ed39a7-ace0-44d5-a4a5-6f20940d5997 service nova] Creating event network-vif-deleted:b4168fa6-3b0a-4107-83cf-7d31f29910d4 for instance fbef28bb-3cdf-4a27-8493-add7806af828 on hv-cinder-84630 Aug 27 02:24:19.688316 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [req-6afbc374-07cd-4458-91ba-d6d66ac97f7f req-c3ed39a7-ace0-44d5-a4a5-6f20940d5997 service nova] 10.222.0.52 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.034096 Aug 27 02:24:19.689412 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 126/248] 10.222.0.52 () {64 vars in 1317 bytes} [Sat Aug 27 02:24:19 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 36 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 27 02:24:20.330549 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-69db3df0-e99b-403e-86d1-1a893ee7d3f8 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Calling method '>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:24:20.331768 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-69db3df0-e99b-403e-86d1-1a893ee7d3f8 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] [instance: 05bed8e9-f530-418a-9819-4f65dabdcda0] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:24:20.339086 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-69db3df0-e99b-403e-86d1-1a893ee7d3f8 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:24:20.339569 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-69db3df0-e99b-403e-86d1-1a893ee7d3f8 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:24:20.340253 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-69db3df0-e99b-403e-86d1-1a893ee7d3f8 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:24:20.340968 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-69db3df0-e99b-403e-86d1-1a893ee7d3f8 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:24:20.341461 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-69db3df0-e99b-403e-86d1-1a893ee7d3f8 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:24:20.341851 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-69db3df0-e99b-403e-86d1-1a893ee7d3f8 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:24:20.400049 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-349aacf2-4217-4f4a-a510-e535959b89c4 tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:24:20.400659 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-349aacf2-4217-4f4a-a510-e535959b89c4 tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] [instance: fbef28bb-3cdf-4a27-8493-add7806af828] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:24:20.407592 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-349aacf2-4217-4f4a-a510-e535959b89c4 tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:24:20.408406 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-349aacf2-4217-4f4a-a510-e535959b89c4 tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:24:20.409269 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-349aacf2-4217-4f4a-a510-e535959b89c4 tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:24:20.410682 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-349aacf2-4217-4f4a-a510-e535959b89c4 tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:24:20.411550 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-349aacf2-4217-4f4a-a510-e535959b89c4 tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:24:20.412193 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-349aacf2-4217-4f4a-a510-e535959b89c4 tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:24:20.496631 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-349aacf2-4217-4f4a-a510-e535959b89c4 tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '55640f750d1b49be9c3f79501a63c1ab', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7ef4b44a3f94d599eacce269b06594c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:24:20.506991 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-349aacf2-4217-4f4a-a510-e535959b89c4 tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] 10.222.0.52 "GET /compute/v2.1/servers/fbef28bb-3cdf-4a27-8493-add7806af828" status: 200 len: 1345 microversion: 2.1 time: 0.111375 Aug 27 02:24:20.508202 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 127/249] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:24:20 2022] GET /compute/v2.1/servers/fbef28bb-3cdf-4a27-8493-add7806af828 => generated 1345 bytes in 114 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:24:20.526047 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-69db3df0-e99b-403e-86d1-1a893ee7d3f8 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd945d512ae034ec08838d0a3afde2429', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '880ac4003248441c809a76cfd1b221eb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:24:20.533856 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-69db3df0-e99b-403e-86d1-1a893ee7d3f8 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] 10.222.0.52 "GET /compute/v2.1/servers/05bed8e9-f530-418a-9819-4f65dabdcda0" status: 200 len: 1521 microversion: 2.1 time: 0.207181 Aug 27 02:24:20.534507 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 123/250] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:24:20 2022] GET /compute/v2.1/servers/05bed8e9-f530-418a-9819-4f65dabdcda0 => generated 1521 bytes in 208 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:24:21.533916 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-1aa0c9be-6b5d-4c48-b384-d0e88fe6a27c tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:24:21.534839 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-1aa0c9be-6b5d-4c48-b384-d0e88fe6a27c tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] [instance: fbef28bb-3cdf-4a27-8493-add7806af828] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:24:21.542373 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-1aa0c9be-6b5d-4c48-b384-d0e88fe6a27c tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:24:21.542843 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-1aa0c9be-6b5d-4c48-b384-d0e88fe6a27c tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:24:21.543320 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-1aa0c9be-6b5d-4c48-b384-d0e88fe6a27c tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:24:21.544166 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-1aa0c9be-6b5d-4c48-b384-d0e88fe6a27c tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:24:21.544715 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-1aa0c9be-6b5d-4c48-b384-d0e88fe6a27c tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:24:21.545166 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-1aa0c9be-6b5d-4c48-b384-d0e88fe6a27c tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:24:21.562368 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.wsgi [None req-1aa0c9be-6b5d-4c48-b384-d0e88fe6a27c tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] HTTP exception thrown: Instance fbef28bb-3cdf-4a27-8493-add7806af828 could not be found. Aug 27 02:24:21.565155 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-6094faf5-34b8-4e71-a3ae-98fbbab53d03 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Calling method '>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:24:21.567658 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-6094faf5-34b8-4e71-a3ae-98fbbab53d03 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] [instance: 05bed8e9-f530-418a-9819-4f65dabdcda0] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:24:21.568515 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-1aa0c9be-6b5d-4c48-b384-d0e88fe6a27c tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Request method failure captured: Aug 27 02:24:21.568515 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: request: GET /compute/v2.1/servers/fbef28bb-3cdf-4a27-8493-add7806af828 HTTP/1.1 Aug 27 02:24:21.568515 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: Accept: application/json Aug 27 02:24:21.568515 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: Accept-Encoding: identity Aug 27 02:24:21.568515 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: Connection: close Aug 27 02:24:21.568515 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: Content-Length: 0 Aug 27 02:24:21.568515 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: Content-Type: application/json Aug 27 02:24:21.568515 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: Host: 10.222.0.52 Aug 27 02:24:21.568515 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: Openstack-System-Scope: None Aug 27 02:24:21.568515 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: User-Agent: python-urllib3/1.26.11 Aug 27 02:24:21.568515 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-Auth-Token: gAAAAABjCX_qzJypsLtNdFaRxHfyf4YzZXOIeCaHBD45kqs0F-zYHAFN3eUtzuHa6A1bPL7qXuQ8CSIyIl6kUz78R8M8sK9yRU4dc-Z4m2mqCHemycgFBIc3LNSxqguqAIQOSOJ1YioTrwVXByTFQrAKhARXfrVLFgNwxXO2H7V_NlBoJ2aS5lo Aug 27 02:24:21.568515 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-Domain-Id: None Aug 27 02:24:21.568515 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-Domain-Name: None Aug 27 02:24:21.568515 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-Identity-Status: Confirmed Aug 27 02:24:21.568515 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-Is-Admin-Project: True Aug 27 02:24:21.568515 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-Project-Domain-Id: default Aug 27 02:24:21.568515 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-Project-Domain-Name: Default Aug 27 02:24:21.568515 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-Project-Id: f7ef4b44a3f94d599eacce269b06594c Aug 27 02:24:21.568515 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-Project-Name: tempest-DeleteServersTestJSON-1052521029 Aug 27 02:24:21.568515 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-Role: reader,member Aug 27 02:24:21.568515 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-Roles: reader,member Aug 27 02:24:21.570814 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-Service-Catalog: [{"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.52:8080", "publicURL": "http://10.222.0.52:8080/v1/AUTH_f7ef4b44a3f94d599eacce269b06594c"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52:9696/networking"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/compute/v2/f7ef4b44a3f94d599eacce269b06594c"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/image"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/volume/v3/f7ef4b44a3f94d599eacce269b06594c"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/identity"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/volume/v3/f7ef4b44a3f94d599eacce269b06594c"}]}] Aug 27 02:24:21.570814 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-Tenant: tempest-DeleteServersTestJSON-1052521029 Aug 27 02:24:21.570814 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-Tenant-Id: f7ef4b44a3f94d599eacce269b06594c Aug 27 02:24:21.570814 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-Tenant-Name: tempest-DeleteServersTestJSON-1052521029 Aug 27 02:24:21.570814 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-User: tempest-DeleteServersTestJSON-1052521029-project Aug 27 02:24:21.570814 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-User-Domain-Id: default Aug 27 02:24:21.570814 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-User-Domain-Name: Default Aug 27 02:24:21.570814 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-User-Id: 55640f750d1b49be9c3f79501a63c1ab Aug 27 02:24:21.570814 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-User-Name: tempest-DeleteServersTestJSON-1052521029-project Aug 27 02:24:21.570814 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: method: > Aug 27 02:24:21.570814 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: exception: Instance fbef28bb-3cdf-4a27-8493-add7806af828 could not be found. Aug 27 02:24:21.570814 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: action_args: {'id': 'fbef28bb-3cdf-4a27-8493-add7806af828'} Aug 27 02:24:21.572732 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Aug 27 02:24:21.572732 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 27 02:24:21.572732 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Aug 27 02:24:21.572732 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Aug 27 02:24:21.572732 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Aug 27 02:24:21.572732 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Aug 27 02:24:21.572732 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Aug 27 02:24:21.572732 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Aug 27 02:24:21.572732 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Aug 27 02:24:21.572732 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Aug 27 02:24:21.572732 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance fbef28bb-3cdf-4a27-8493-add7806af828 could not be found. Aug 27 02:24:21.572732 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi Aug 27 02:24:21.572732 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 27 02:24:21.572732 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi Aug 27 02:24:21.572732 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 27 02:24:21.572732 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 27 02:24:21.572732 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 27 02:24:21.572732 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Aug 27 02:24:21.572732 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Aug 27 02:24:21.572732 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Aug 27 02:24:21.576819 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Aug 27 02:24:21.576819 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Aug 27 02:24:21.576819 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi instance = self._get_server( Aug 27 02:24:21.576819 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Aug 27 02:24:21.576819 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Aug 27 02:24:21.576819 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Aug 27 02:24:21.576819 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Aug 27 02:24:21.576819 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance fbef28bb-3cdf-4a27-8493-add7806af828 could not be found. Aug 27 02:24:21.576819 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi Aug 27 02:24:21.576819 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 27 02:24:21.576819 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi Aug 27 02:24:21.576819 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 27 02:24:21.576819 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 27 02:24:21.576819 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 27 02:24:21.576819 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Aug 27 02:24:21.576819 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Aug 27 02:24:21.576819 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance fbef28bb-3cdf-4a27-8493-add7806af828 could not be found. Aug 27 02:24:21.576819 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi Aug 27 02:24:21.581296 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-1aa0c9be-6b5d-4c48-b384-d0e88fe6a27c tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Returning 404 to user: Instance fbef28bb-3cdf-4a27-8493-add7806af828 could not be found. {{(pid=120863) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Aug 27 02:24:21.581296 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-1aa0c9be-6b5d-4c48-b384-d0e88fe6a27c tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] 10.222.0.52 "GET /compute/v2.1/servers/fbef28bb-3cdf-4a27-8493-add7806af828" status: 404 len: 111 microversion: 2.1 time: 0.041566 Aug 27 02:24:21.581296 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 128/251] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:24:21 2022] GET /compute/v2.1/servers/fbef28bb-3cdf-4a27-8493-add7806af828 => generated 111 bytes in 43 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Aug 27 02:24:21.594055 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-6094faf5-34b8-4e71-a3ae-98fbbab53d03 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:24:21.595485 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-6094faf5-34b8-4e71-a3ae-98fbbab53d03 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:24:21.596461 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-6094faf5-34b8-4e71-a3ae-98fbbab53d03 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:24:21.598124 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-6094faf5-34b8-4e71-a3ae-98fbbab53d03 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:24:21.599166 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-6094faf5-34b8-4e71-a3ae-98fbbab53d03 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:24:21.600178 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-6094faf5-34b8-4e71-a3ae-98fbbab53d03 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:24:21.737564 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-a55e325c-4272-4ca8-a992-a0e02965a5d0 tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:24:21.740429 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-a55e325c-4272-4ca8-a992-a0e02965a5d0 tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] [instance: fbef28bb-3cdf-4a27-8493-add7806af828] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:24:21.751059 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-a55e325c-4272-4ca8-a992-a0e02965a5d0 tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:24:21.751791 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-a55e325c-4272-4ca8-a992-a0e02965a5d0 tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:24:21.752719 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-a55e325c-4272-4ca8-a992-a0e02965a5d0 tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:24:21.754323 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-a55e325c-4272-4ca8-a992-a0e02965a5d0 tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:24:21.755039 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-a55e325c-4272-4ca8-a992-a0e02965a5d0 tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:24:21.755936 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-a55e325c-4272-4ca8-a992-a0e02965a5d0 tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:24:21.781734 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.wsgi [None req-a55e325c-4272-4ca8-a992-a0e02965a5d0 tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] HTTP exception thrown: Instance fbef28bb-3cdf-4a27-8493-add7806af828 could not be found. Aug 27 02:24:21.785673 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-a55e325c-4272-4ca8-a992-a0e02965a5d0 tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Request method failure captured: Aug 27 02:24:21.785673 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: request: GET /compute/v2.1/servers/fbef28bb-3cdf-4a27-8493-add7806af828/os-volume_attachments HTTP/1.1 Aug 27 02:24:21.785673 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: Accept: application/json Aug 27 02:24:21.785673 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: Accept-Encoding: identity Aug 27 02:24:21.785673 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: Connection: close Aug 27 02:24:21.785673 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: Content-Length: 0 Aug 27 02:24:21.785673 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: Content-Type: application/json Aug 27 02:24:21.785673 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: Host: 10.222.0.52 Aug 27 02:24:21.785673 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: Openstack-System-Scope: None Aug 27 02:24:21.785673 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: User-Agent: python-urllib3/1.26.11 Aug 27 02:24:21.785673 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-Auth-Token: gAAAAABjCX_qzJypsLtNdFaRxHfyf4YzZXOIeCaHBD45kqs0F-zYHAFN3eUtzuHa6A1bPL7qXuQ8CSIyIl6kUz78R8M8sK9yRU4dc-Z4m2mqCHemycgFBIc3LNSxqguqAIQOSOJ1YioTrwVXByTFQrAKhARXfrVLFgNwxXO2H7V_NlBoJ2aS5lo Aug 27 02:24:21.785673 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-Domain-Id: None Aug 27 02:24:21.785673 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-Domain-Name: None Aug 27 02:24:21.785673 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-Identity-Status: Confirmed Aug 27 02:24:21.785673 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-Is-Admin-Project: True Aug 27 02:24:21.785673 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-Project-Domain-Id: default Aug 27 02:24:21.785673 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-Project-Domain-Name: Default Aug 27 02:24:21.785673 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-Project-Id: f7ef4b44a3f94d599eacce269b06594c Aug 27 02:24:21.785673 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-Project-Name: tempest-DeleteServersTestJSON-1052521029 Aug 27 02:24:21.785673 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-Role: reader,member Aug 27 02:24:21.785673 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-Roles: reader,member Aug 27 02:24:21.787121 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-Service-Catalog: [{"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.52:8080", "publicURL": "http://10.222.0.52:8080/v1/AUTH_f7ef4b44a3f94d599eacce269b06594c"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52:9696/networking"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/compute/v2/f7ef4b44a3f94d599eacce269b06594c"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/image"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/volume/v3/f7ef4b44a3f94d599eacce269b06594c"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/identity"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/volume/v3/f7ef4b44a3f94d599eacce269b06594c"}]}] Aug 27 02:24:21.787121 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-Tenant: tempest-DeleteServersTestJSON-1052521029 Aug 27 02:24:21.787121 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-Tenant-Id: f7ef4b44a3f94d599eacce269b06594c Aug 27 02:24:21.787121 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-Tenant-Name: tempest-DeleteServersTestJSON-1052521029 Aug 27 02:24:21.787121 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-User: tempest-DeleteServersTestJSON-1052521029-project Aug 27 02:24:21.787121 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-User-Domain-Id: default Aug 27 02:24:21.787121 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-User-Domain-Name: Default Aug 27 02:24:21.787121 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-User-Id: 55640f750d1b49be9c3f79501a63c1ab Aug 27 02:24:21.787121 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-User-Name: tempest-DeleteServersTestJSON-1052521029-project Aug 27 02:24:21.787121 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: method: > Aug 27 02:24:21.787121 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: exception: Instance fbef28bb-3cdf-4a27-8493-add7806af828 could not be found. Aug 27 02:24:21.787121 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: action_args: {'server_id': 'fbef28bb-3cdf-4a27-8493-add7806af828'} Aug 27 02:24:21.787959 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Aug 27 02:24:21.787959 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 27 02:24:21.787959 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Aug 27 02:24:21.787959 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Aug 27 02:24:21.787959 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Aug 27 02:24:21.787959 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Aug 27 02:24:21.787959 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Aug 27 02:24:21.787959 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Aug 27 02:24:21.787959 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Aug 27 02:24:21.787959 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Aug 27 02:24:21.787959 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance fbef28bb-3cdf-4a27-8493-add7806af828 could not be found. Aug 27 02:24:21.787959 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi Aug 27 02:24:21.787959 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 27 02:24:21.787959 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi Aug 27 02:24:21.787959 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 27 02:24:21.787959 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 27 02:24:21.787959 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 27 02:24:21.787959 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Aug 27 02:24:21.787959 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Aug 27 02:24:21.787959 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Aug 27 02:24:21.789484 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Aug 27 02:24:21.789484 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 192, in wrapper Aug 27 02:24:21.789484 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Aug 27 02:24:21.789484 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 192, in wrapper Aug 27 02:24:21.789484 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Aug 27 02:24:21.789484 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 303, in index Aug 27 02:24:21.789484 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, server_id) Aug 27 02:24:21.789484 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Aug 27 02:24:21.789484 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Aug 27 02:24:21.789484 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance fbef28bb-3cdf-4a27-8493-add7806af828 could not be found. Aug 27 02:24:21.789484 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi Aug 27 02:24:21.789484 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 27 02:24:21.789484 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi Aug 27 02:24:21.789484 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 27 02:24:21.789484 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 27 02:24:21.789484 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 27 02:24:21.789484 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Aug 27 02:24:21.789484 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Aug 27 02:24:21.789484 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance fbef28bb-3cdf-4a27-8493-add7806af828 could not be found. Aug 27 02:24:21.789484 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi Aug 27 02:24:21.793769 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-a55e325c-4272-4ca8-a992-a0e02965a5d0 tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Returning 404 to user: Instance fbef28bb-3cdf-4a27-8493-add7806af828 could not be found. {{(pid=120863) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Aug 27 02:24:21.794860 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-a55e325c-4272-4ca8-a992-a0e02965a5d0 tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] 10.222.0.52 "GET /compute/v2.1/servers/fbef28bb-3cdf-4a27-8493-add7806af828/os-volume_attachments" status: 404 len: 111 microversion: 2.1 time: 0.061692 Aug 27 02:24:21.795759 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 129/252] 10.222.0.52 () {58 vars in 1302 bytes} [Sat Aug 27 02:24:21 2022] GET /compute/v2.1/servers/fbef28bb-3cdf-4a27-8493-add7806af828/os-volume_attachments => generated 111 bytes in 63 msecs (HTTP/1.1 404) 9 headers in 379 bytes (2 switches on core 0) Aug 27 02:24:21.888803 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-6094faf5-34b8-4e71-a3ae-98fbbab53d03 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd945d512ae034ec08838d0a3afde2429', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '880ac4003248441c809a76cfd1b221eb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:24:21.899038 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-6094faf5-34b8-4e71-a3ae-98fbbab53d03 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] 10.222.0.52 "GET /compute/v2.1/servers/05bed8e9-f530-418a-9819-4f65dabdcda0" status: 200 len: 1521 microversion: 2.1 time: 0.342290 Aug 27 02:24:21.899916 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 124/253] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:24:21 2022] GET /compute/v2.1/servers/05bed8e9-f530-418a-9819-4f65dabdcda0 => generated 1521 bytes in 347 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:24:22.925329 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-152e11dd-7896-42e0-b239-fd08a7615bb3 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:24:22.931621 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-152e11dd-7896-42e0-b239-fd08a7615bb3 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] [instance: 05bed8e9-f530-418a-9819-4f65dabdcda0] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:24:22.940114 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-152e11dd-7896-42e0-b239-fd08a7615bb3 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:24:22.940671 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-152e11dd-7896-42e0-b239-fd08a7615bb3 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:24:22.941668 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-152e11dd-7896-42e0-b239-fd08a7615bb3 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:24:22.942624 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-152e11dd-7896-42e0-b239-fd08a7615bb3 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:24:22.943160 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-152e11dd-7896-42e0-b239-fd08a7615bb3 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:24:22.943645 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-152e11dd-7896-42e0-b239-fd08a7615bb3 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:24:23.093021 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-8b61de24-a28f-4b60-b202-5a25aa67d86e tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Calling method '>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:24:23.094519 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-8b61de24-a28f-4b60-b202-5a25aa67d86e tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] [instance: fbef28bb-3cdf-4a27-8493-add7806af828] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:24:23.102182 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-8b61de24-a28f-4b60-b202-5a25aa67d86e tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:24:23.103022 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-8b61de24-a28f-4b60-b202-5a25aa67d86e tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:24:23.103958 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-8b61de24-a28f-4b60-b202-5a25aa67d86e tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:24:23.104934 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-8b61de24-a28f-4b60-b202-5a25aa67d86e tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:24:23.105640 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-8b61de24-a28f-4b60-b202-5a25aa67d86e tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:24:23.106277 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-8b61de24-a28f-4b60-b202-5a25aa67d86e tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:24:23.122260 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.wsgi [None req-8b61de24-a28f-4b60-b202-5a25aa67d86e tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] HTTP exception thrown: Instance fbef28bb-3cdf-4a27-8493-add7806af828 could not be found. Aug 27 02:24:23.127235 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-8b61de24-a28f-4b60-b202-5a25aa67d86e tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Request method failure captured: Aug 27 02:24:23.127235 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: request: DELETE /compute/v2.1/servers/fbef28bb-3cdf-4a27-8493-add7806af828 HTTP/1.1 Aug 27 02:24:23.127235 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: Accept: application/json Aug 27 02:24:23.127235 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: Accept-Encoding: identity Aug 27 02:24:23.127235 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: Connection: close Aug 27 02:24:23.127235 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: Content-Length: 0 Aug 27 02:24:23.127235 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: Content-Type: application/json Aug 27 02:24:23.127235 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: Host: 10.222.0.52 Aug 27 02:24:23.127235 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: Openstack-System-Scope: None Aug 27 02:24:23.127235 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: User-Agent: python-urllib3/1.26.11 Aug 27 02:24:23.127235 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-Auth-Token: gAAAAABjCX_qzJypsLtNdFaRxHfyf4YzZXOIeCaHBD45kqs0F-zYHAFN3eUtzuHa6A1bPL7qXuQ8CSIyIl6kUz78R8M8sK9yRU4dc-Z4m2mqCHemycgFBIc3LNSxqguqAIQOSOJ1YioTrwVXByTFQrAKhARXfrVLFgNwxXO2H7V_NlBoJ2aS5lo Aug 27 02:24:23.127235 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-Domain-Id: None Aug 27 02:24:23.127235 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-Domain-Name: None Aug 27 02:24:23.127235 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-Identity-Status: Confirmed Aug 27 02:24:23.127235 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-Is-Admin-Project: True Aug 27 02:24:23.127235 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-Project-Domain-Id: default Aug 27 02:24:23.127235 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-Project-Domain-Name: Default Aug 27 02:24:23.127235 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-Project-Id: f7ef4b44a3f94d599eacce269b06594c Aug 27 02:24:23.127235 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-Project-Name: tempest-DeleteServersTestJSON-1052521029 Aug 27 02:24:23.127235 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-Role: reader,member Aug 27 02:24:23.127235 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-Roles: reader,member Aug 27 02:24:23.128423 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-Service-Catalog: [{"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.52:8080", "publicURL": "http://10.222.0.52:8080/v1/AUTH_f7ef4b44a3f94d599eacce269b06594c"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52:9696/networking"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/compute/v2/f7ef4b44a3f94d599eacce269b06594c"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/image"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/volume/v3/f7ef4b44a3f94d599eacce269b06594c"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/identity"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/volume/v3/f7ef4b44a3f94d599eacce269b06594c"}]}] Aug 27 02:24:23.128423 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-Tenant: tempest-DeleteServersTestJSON-1052521029 Aug 27 02:24:23.128423 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-Tenant-Id: f7ef4b44a3f94d599eacce269b06594c Aug 27 02:24:23.128423 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-Tenant-Name: tempest-DeleteServersTestJSON-1052521029 Aug 27 02:24:23.128423 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-User: tempest-DeleteServersTestJSON-1052521029-project Aug 27 02:24:23.128423 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-User-Domain-Id: default Aug 27 02:24:23.128423 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-User-Domain-Name: Default Aug 27 02:24:23.128423 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-User-Id: 55640f750d1b49be9c3f79501a63c1ab Aug 27 02:24:23.128423 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-User-Name: tempest-DeleteServersTestJSON-1052521029-project Aug 27 02:24:23.128423 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: method: > Aug 27 02:24:23.128423 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: exception: Instance fbef28bb-3cdf-4a27-8493-add7806af828 could not be found. Aug 27 02:24:23.128423 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: action_args: {'id': 'fbef28bb-3cdf-4a27-8493-add7806af828'} Aug 27 02:24:23.129235 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Aug 27 02:24:23.129235 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 27 02:24:23.129235 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Aug 27 02:24:23.129235 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Aug 27 02:24:23.129235 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Aug 27 02:24:23.129235 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Aug 27 02:24:23.129235 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Aug 27 02:24:23.129235 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Aug 27 02:24:23.129235 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Aug 27 02:24:23.129235 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Aug 27 02:24:23.129235 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance fbef28bb-3cdf-4a27-8493-add7806af828 could not be found. Aug 27 02:24:23.129235 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi Aug 27 02:24:23.129235 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 27 02:24:23.129235 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi Aug 27 02:24:23.129235 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 27 02:24:23.129235 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 27 02:24:23.129235 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 27 02:24:23.129235 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Aug 27 02:24:23.129235 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Aug 27 02:24:23.129235 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Aug 27 02:24:23.130561 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Aug 27 02:24:23.130561 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Aug 27 02:24:23.130561 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Aug 27 02:24:23.130561 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Aug 27 02:24:23.130561 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Aug 27 02:24:23.130561 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Aug 27 02:24:23.130561 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Aug 27 02:24:23.130561 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Aug 27 02:24:23.130561 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Aug 27 02:24:23.130561 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance fbef28bb-3cdf-4a27-8493-add7806af828 could not be found. Aug 27 02:24:23.130561 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi Aug 27 02:24:23.130561 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 27 02:24:23.130561 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi Aug 27 02:24:23.130561 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 27 02:24:23.130561 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 27 02:24:23.130561 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 27 02:24:23.130561 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Aug 27 02:24:23.130561 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Aug 27 02:24:23.130561 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance fbef28bb-3cdf-4a27-8493-add7806af828 could not be found. Aug 27 02:24:23.130561 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi Aug 27 02:24:23.135016 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-8b61de24-a28f-4b60-b202-5a25aa67d86e tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Returning 404 to user: Instance fbef28bb-3cdf-4a27-8493-add7806af828 could not be found. {{(pid=120864) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Aug 27 02:24:23.136180 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-8b61de24-a28f-4b60-b202-5a25aa67d86e tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] 10.222.0.52 "DELETE /compute/v2.1/servers/fbef28bb-3cdf-4a27-8493-add7806af828" status: 404 len: 111 microversion: 2.1 time: 0.049422 Aug 27 02:24:23.136956 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 125/254] 10.222.0.52 () {58 vars in 1239 bytes} [Sat Aug 27 02:24:23 2022] DELETE /compute/v2.1/servers/fbef28bb-3cdf-4a27-8493-add7806af828 => generated 111 bytes in 51 msecs (HTTP/1.1 404) 9 headers in 379 bytes (2 switches on core 0) Aug 27 02:24:23.145472 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-e01e306e-89be-4d76-a1c3-6b1a771605c4 tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Calling method '>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:24:23.146347 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-e01e306e-89be-4d76-a1c3-6b1a771605c4 tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] [instance: fbef28bb-3cdf-4a27-8493-add7806af828] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:24:23.152536 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-e01e306e-89be-4d76-a1c3-6b1a771605c4 tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:24:23.153339 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-e01e306e-89be-4d76-a1c3-6b1a771605c4 tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:24:23.154289 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-152e11dd-7896-42e0-b239-fd08a7615bb3 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd945d512ae034ec08838d0a3afde2429', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '880ac4003248441c809a76cfd1b221eb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:24:23.154409 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-e01e306e-89be-4d76-a1c3-6b1a771605c4 tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:24:23.155378 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-e01e306e-89be-4d76-a1c3-6b1a771605c4 tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:24:23.156373 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-e01e306e-89be-4d76-a1c3-6b1a771605c4 tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:24:23.157289 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-e01e306e-89be-4d76-a1c3-6b1a771605c4 tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:24:23.163886 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-152e11dd-7896-42e0-b239-fd08a7615bb3 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] 10.222.0.52 "GET /compute/v2.1/servers/05bed8e9-f530-418a-9819-4f65dabdcda0" status: 200 len: 1521 microversion: 2.1 time: 0.245018 Aug 27 02:24:23.165045 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 130/255] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:24:22 2022] GET /compute/v2.1/servers/05bed8e9-f530-418a-9819-4f65dabdcda0 => generated 1521 bytes in 248 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 27 02:24:23.174388 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.wsgi [None req-e01e306e-89be-4d76-a1c3-6b1a771605c4 tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] HTTP exception thrown: Instance fbef28bb-3cdf-4a27-8493-add7806af828 could not be found. Aug 27 02:24:23.176499 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-e01e306e-89be-4d76-a1c3-6b1a771605c4 tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Request method failure captured: Aug 27 02:24:23.176499 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: request: GET /compute/v2.1/servers/fbef28bb-3cdf-4a27-8493-add7806af828 HTTP/1.1 Aug 27 02:24:23.176499 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: Accept: application/json Aug 27 02:24:23.176499 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: Accept-Encoding: identity Aug 27 02:24:23.176499 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: Connection: close Aug 27 02:24:23.176499 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: Content-Length: 0 Aug 27 02:24:23.176499 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: Content-Type: application/json Aug 27 02:24:23.176499 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: Host: 10.222.0.52 Aug 27 02:24:23.176499 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: Openstack-System-Scope: None Aug 27 02:24:23.176499 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: User-Agent: python-urllib3/1.26.11 Aug 27 02:24:23.176499 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-Auth-Token: gAAAAABjCX_qzJypsLtNdFaRxHfyf4YzZXOIeCaHBD45kqs0F-zYHAFN3eUtzuHa6A1bPL7qXuQ8CSIyIl6kUz78R8M8sK9yRU4dc-Z4m2mqCHemycgFBIc3LNSxqguqAIQOSOJ1YioTrwVXByTFQrAKhARXfrVLFgNwxXO2H7V_NlBoJ2aS5lo Aug 27 02:24:23.176499 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-Domain-Id: None Aug 27 02:24:23.176499 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-Domain-Name: None Aug 27 02:24:23.176499 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-Identity-Status: Confirmed Aug 27 02:24:23.176499 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-Is-Admin-Project: True Aug 27 02:24:23.176499 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-Project-Domain-Id: default Aug 27 02:24:23.176499 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-Project-Domain-Name: Default Aug 27 02:24:23.176499 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-Project-Id: f7ef4b44a3f94d599eacce269b06594c Aug 27 02:24:23.176499 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-Project-Name: tempest-DeleteServersTestJSON-1052521029 Aug 27 02:24:23.176499 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-Role: reader,member Aug 27 02:24:23.176499 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-Roles: reader,member Aug 27 02:24:23.178078 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-Service-Catalog: [{"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.52:8080", "publicURL": "http://10.222.0.52:8080/v1/AUTH_f7ef4b44a3f94d599eacce269b06594c"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52:9696/networking"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/compute/v2/f7ef4b44a3f94d599eacce269b06594c"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/image"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/volume/v3/f7ef4b44a3f94d599eacce269b06594c"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/identity"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/volume/v3/f7ef4b44a3f94d599eacce269b06594c"}]}] Aug 27 02:24:23.178078 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-Tenant: tempest-DeleteServersTestJSON-1052521029 Aug 27 02:24:23.178078 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-Tenant-Id: f7ef4b44a3f94d599eacce269b06594c Aug 27 02:24:23.178078 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-Tenant-Name: tempest-DeleteServersTestJSON-1052521029 Aug 27 02:24:23.178078 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-User: tempest-DeleteServersTestJSON-1052521029-project Aug 27 02:24:23.178078 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-User-Domain-Id: default Aug 27 02:24:23.178078 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-User-Domain-Name: Default Aug 27 02:24:23.178078 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-User-Id: 55640f750d1b49be9c3f79501a63c1ab Aug 27 02:24:23.178078 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-User-Name: tempest-DeleteServersTestJSON-1052521029-project Aug 27 02:24:23.178078 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: method: > Aug 27 02:24:23.178078 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: exception: Instance fbef28bb-3cdf-4a27-8493-add7806af828 could not be found. Aug 27 02:24:23.178078 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: action_args: {'id': 'fbef28bb-3cdf-4a27-8493-add7806af828'} Aug 27 02:24:23.178806 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Aug 27 02:24:23.178806 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 27 02:24:23.178806 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Aug 27 02:24:23.178806 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Aug 27 02:24:23.178806 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Aug 27 02:24:23.178806 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Aug 27 02:24:23.178806 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Aug 27 02:24:23.178806 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Aug 27 02:24:23.178806 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Aug 27 02:24:23.178806 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Aug 27 02:24:23.178806 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance fbef28bb-3cdf-4a27-8493-add7806af828 could not be found. Aug 27 02:24:23.178806 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi Aug 27 02:24:23.178806 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 27 02:24:23.178806 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi Aug 27 02:24:23.178806 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 27 02:24:23.178806 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 27 02:24:23.178806 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 27 02:24:23.178806 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Aug 27 02:24:23.178806 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Aug 27 02:24:23.178806 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Aug 27 02:24:23.179814 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Aug 27 02:24:23.179814 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Aug 27 02:24:23.179814 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi instance = self._get_server( Aug 27 02:24:23.179814 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Aug 27 02:24:23.179814 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Aug 27 02:24:23.179814 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Aug 27 02:24:23.179814 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Aug 27 02:24:23.179814 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance fbef28bb-3cdf-4a27-8493-add7806af828 could not be found. Aug 27 02:24:23.179814 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi Aug 27 02:24:23.179814 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 27 02:24:23.179814 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi Aug 27 02:24:23.179814 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 27 02:24:23.179814 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 27 02:24:23.179814 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 27 02:24:23.179814 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Aug 27 02:24:23.179814 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Aug 27 02:24:23.179814 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance fbef28bb-3cdf-4a27-8493-add7806af828 could not be found. Aug 27 02:24:23.179814 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi Aug 27 02:24:23.182100 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-e01e306e-89be-4d76-a1c3-6b1a771605c4 tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] Returning 404 to user: Instance fbef28bb-3cdf-4a27-8493-add7806af828 could not be found. {{(pid=120864) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Aug 27 02:24:23.183025 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-e01e306e-89be-4d76-a1c3-6b1a771605c4 tempest-DeleteServersTestJSON-1052521029 tempest-DeleteServersTestJSON-1052521029-project] 10.222.0.52 "GET /compute/v2.1/servers/fbef28bb-3cdf-4a27-8493-add7806af828" status: 404 len: 111 microversion: 2.1 time: 0.041599 Aug 27 02:24:23.184066 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 126/256] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:24:23 2022] GET /compute/v2.1/servers/fbef28bb-3cdf-4a27-8493-add7806af828 => generated 111 bytes in 43 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Aug 27 02:24:24.187620 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-1d0fe401-dc66-4f85-b1d3-cda3ca073e74 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:24:24.188229 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-1d0fe401-dc66-4f85-b1d3-cda3ca073e74 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] [instance: 05bed8e9-f530-418a-9819-4f65dabdcda0] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:24:24.190808 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG dbcounter [-] [120863] Writing DB stats nova_api:SELECT=47,nova_api:UPDATE=1 {{(pid=120863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 27 02:24:24.201948 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-1d0fe401-dc66-4f85-b1d3-cda3ca073e74 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:24:24.202438 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-1d0fe401-dc66-4f85-b1d3-cda3ca073e74 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:24:24.202872 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-1d0fe401-dc66-4f85-b1d3-cda3ca073e74 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:24:24.203578 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-1d0fe401-dc66-4f85-b1d3-cda3ca073e74 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:24:24.204032 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-1d0fe401-dc66-4f85-b1d3-cda3ca073e74 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:24:24.204453 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-1d0fe401-dc66-4f85-b1d3-cda3ca073e74 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:24:24.466132 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-1d0fe401-dc66-4f85-b1d3-cda3ca073e74 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd945d512ae034ec08838d0a3afde2429', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '880ac4003248441c809a76cfd1b221eb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:24:24.477777 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-1d0fe401-dc66-4f85-b1d3-cda3ca073e74 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] 10.222.0.52 "GET /compute/v2.1/servers/05bed8e9-f530-418a-9819-4f65dabdcda0" status: 200 len: 1521 microversion: 2.1 time: 0.293844 Aug 27 02:24:24.479739 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 131/257] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:24:24 2022] GET /compute/v2.1/servers/05bed8e9-f530-418a-9819-4f65dabdcda0 => generated 1521 bytes in 296 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:24:25.501894 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-4c1c9f71-34b7-4ba1-8839-aceca01a2675 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Calling method '>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:24:25.503639 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-4c1c9f71-34b7-4ba1-8839-aceca01a2675 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] [instance: 05bed8e9-f530-418a-9819-4f65dabdcda0] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:24:25.514273 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-4c1c9f71-34b7-4ba1-8839-aceca01a2675 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:24:25.515376 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-4c1c9f71-34b7-4ba1-8839-aceca01a2675 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:24:25.517424 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-4c1c9f71-34b7-4ba1-8839-aceca01a2675 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:24:25.520192 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-4c1c9f71-34b7-4ba1-8839-aceca01a2675 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:24:25.522047 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-4c1c9f71-34b7-4ba1-8839-aceca01a2675 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:24:25.526520 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-4c1c9f71-34b7-4ba1-8839-aceca01a2675 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:24:25.884199 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-4c1c9f71-34b7-4ba1-8839-aceca01a2675 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd945d512ae034ec08838d0a3afde2429', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '880ac4003248441c809a76cfd1b221eb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:24:25.897487 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-4c1c9f71-34b7-4ba1-8839-aceca01a2675 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] 10.222.0.52 "GET /compute/v2.1/servers/05bed8e9-f530-418a-9819-4f65dabdcda0" status: 200 len: 1521 microversion: 2.1 time: 0.400919 Aug 27 02:24:25.899107 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 127/258] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:24:25 2022] GET /compute/v2.1/servers/05bed8e9-f530-418a-9819-4f65dabdcda0 => generated 1521 bytes in 403 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 27 02:24:26.919252 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-dbc4f2fb-96c1-4bca-ad48-694d12106ea6 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:24:26.919785 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-dbc4f2fb-96c1-4bca-ad48-694d12106ea6 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] [instance: 05bed8e9-f530-418a-9819-4f65dabdcda0] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:24:26.929504 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-dbc4f2fb-96c1-4bca-ad48-694d12106ea6 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:24:26.930005 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-dbc4f2fb-96c1-4bca-ad48-694d12106ea6 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:24:26.930443 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-dbc4f2fb-96c1-4bca-ad48-694d12106ea6 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:24:26.931179 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-dbc4f2fb-96c1-4bca-ad48-694d12106ea6 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:24:26.931610 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-dbc4f2fb-96c1-4bca-ad48-694d12106ea6 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:24:26.937744 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-dbc4f2fb-96c1-4bca-ad48-694d12106ea6 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.006s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:24:27.378957 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-dbc4f2fb-96c1-4bca-ad48-694d12106ea6 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd945d512ae034ec08838d0a3afde2429', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '880ac4003248441c809a76cfd1b221eb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:24:27.415808 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-dbc4f2fb-96c1-4bca-ad48-694d12106ea6 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] 10.222.0.52 "GET /compute/v2.1/servers/05bed8e9-f530-418a-9819-4f65dabdcda0" status: 200 len: 1521 microversion: 2.1 time: 0.493594 Aug 27 02:24:27.431497 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 132/259] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:24:26 2022] GET /compute/v2.1/servers/05bed8e9-f530-418a-9819-4f65dabdcda0 => generated 1521 bytes in 508 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:24:28.446093 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-d87c4a09-80f6-429f-bdd0-71ab43dcc29c tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Calling method '>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:24:28.447550 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-d87c4a09-80f6-429f-bdd0-71ab43dcc29c tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] [instance: 05bed8e9-f530-418a-9819-4f65dabdcda0] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:24:28.454253 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-d87c4a09-80f6-429f-bdd0-71ab43dcc29c tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:24:28.455540 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-d87c4a09-80f6-429f-bdd0-71ab43dcc29c tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:24:28.456348 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-d87c4a09-80f6-429f-bdd0-71ab43dcc29c tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:24:28.457611 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-d87c4a09-80f6-429f-bdd0-71ab43dcc29c tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:24:28.458275 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-d87c4a09-80f6-429f-bdd0-71ab43dcc29c tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:24:28.459039 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-d87c4a09-80f6-429f-bdd0-71ab43dcc29c tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:24:28.485784 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG dbcounter [-] [120864] Writing DB stats nova_cell1:SELECT=100 {{(pid=120864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 27 02:24:28.738230 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-d87c4a09-80f6-429f-bdd0-71ab43dcc29c tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd945d512ae034ec08838d0a3afde2429', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '880ac4003248441c809a76cfd1b221eb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:24:28.761078 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-d87c4a09-80f6-429f-bdd0-71ab43dcc29c tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] 10.222.0.52 "GET /compute/v2.1/servers/05bed8e9-f530-418a-9819-4f65dabdcda0" status: 200 len: 1521 microversion: 2.1 time: 0.319078 Aug 27 02:24:28.761691 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 128/260] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:24:28 2022] GET /compute/v2.1/servers/05bed8e9-f530-418a-9819-4f65dabdcda0 => generated 1521 bytes in 321 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:24:29.783001 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-aac25408-b17c-496e-b280-3b25f964ffde tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:24:29.783610 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-aac25408-b17c-496e-b280-3b25f964ffde tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] [instance: 05bed8e9-f530-418a-9819-4f65dabdcda0] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:24:29.791277 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-aac25408-b17c-496e-b280-3b25f964ffde tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:24:29.792664 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-aac25408-b17c-496e-b280-3b25f964ffde tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:24:29.793329 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-aac25408-b17c-496e-b280-3b25f964ffde tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:24:29.794111 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-aac25408-b17c-496e-b280-3b25f964ffde tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:24:29.794635 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-aac25408-b17c-496e-b280-3b25f964ffde tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:24:29.795131 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-aac25408-b17c-496e-b280-3b25f964ffde tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:24:30.026365 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-aac25408-b17c-496e-b280-3b25f964ffde tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd945d512ae034ec08838d0a3afde2429', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '880ac4003248441c809a76cfd1b221eb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:24:30.035205 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-aac25408-b17c-496e-b280-3b25f964ffde tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] 10.222.0.52 "GET /compute/v2.1/servers/05bed8e9-f530-418a-9819-4f65dabdcda0" status: 200 len: 1521 microversion: 2.1 time: 0.255520 Aug 27 02:24:30.036591 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 133/261] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:24:29 2022] GET /compute/v2.1/servers/05bed8e9-f530-418a-9819-4f65dabdcda0 => generated 1521 bytes in 258 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:24:31.057811 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-58167aea-8934-4ac9-ad14-605a7f8a98ab tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Calling method '>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:24:31.058554 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-58167aea-8934-4ac9-ad14-605a7f8a98ab tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] [instance: 05bed8e9-f530-418a-9819-4f65dabdcda0] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:24:31.064233 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG dbcounter [-] [120864] Writing DB stats nova_api:SELECT=40 {{(pid=120864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 27 02:24:31.065788 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-58167aea-8934-4ac9-ad14-605a7f8a98ab tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:24:31.066463 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-58167aea-8934-4ac9-ad14-605a7f8a98ab tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:24:31.067692 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-58167aea-8934-4ac9-ad14-605a7f8a98ab tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:24:31.068608 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-58167aea-8934-4ac9-ad14-605a7f8a98ab tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:24:31.069233 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-58167aea-8934-4ac9-ad14-605a7f8a98ab tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:24:31.069827 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-58167aea-8934-4ac9-ad14-605a7f8a98ab tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:24:31.249635 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-58167aea-8934-4ac9-ad14-605a7f8a98ab tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd945d512ae034ec08838d0a3afde2429', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '880ac4003248441c809a76cfd1b221eb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:24:31.260323 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-58167aea-8934-4ac9-ad14-605a7f8a98ab tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] 10.222.0.52 "GET /compute/v2.1/servers/05bed8e9-f530-418a-9819-4f65dabdcda0" status: 200 len: 1521 microversion: 2.1 time: 0.205808 Aug 27 02:24:31.261092 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 129/262] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:24:31 2022] GET /compute/v2.1/servers/05bed8e9-f530-418a-9819-4f65dabdcda0 => generated 1521 bytes in 208 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:24:32.290785 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-8edd62ab-67e3-4243-bedf-7b487a91e77c tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:24:32.291421 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-8edd62ab-67e3-4243-bedf-7b487a91e77c tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] [instance: 05bed8e9-f530-418a-9819-4f65dabdcda0] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:24:32.299693 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-8edd62ab-67e3-4243-bedf-7b487a91e77c tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:24:32.300183 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-8edd62ab-67e3-4243-bedf-7b487a91e77c tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:24:32.300992 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-8edd62ab-67e3-4243-bedf-7b487a91e77c tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:24:32.301743 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-8edd62ab-67e3-4243-bedf-7b487a91e77c tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:24:32.302200 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-8edd62ab-67e3-4243-bedf-7b487a91e77c tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:24:32.302625 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-8edd62ab-67e3-4243-bedf-7b487a91e77c tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:24:32.322157 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG dbcounter [-] [120863] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=120863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 27 02:24:32.507035 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-8edd62ab-67e3-4243-bedf-7b487a91e77c tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd945d512ae034ec08838d0a3afde2429', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '880ac4003248441c809a76cfd1b221eb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:24:32.520070 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-8edd62ab-67e3-4243-bedf-7b487a91e77c tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] 10.222.0.52 "GET /compute/v2.1/servers/05bed8e9-f530-418a-9819-4f65dabdcda0" status: 200 len: 1521 microversion: 2.1 time: 0.240329 Aug 27 02:24:32.520991 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 134/263] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:24:32 2022] GET /compute/v2.1/servers/05bed8e9-f530-418a-9819-4f65dabdcda0 => generated 1521 bytes in 242 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:24:33.543187 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-5e6daba6-78f8-465f-b82f-7e0916d555c3 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Calling method '>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:24:33.544526 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-5e6daba6-78f8-465f-b82f-7e0916d555c3 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] [instance: 05bed8e9-f530-418a-9819-4f65dabdcda0] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:24:33.555308 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-5e6daba6-78f8-465f-b82f-7e0916d555c3 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:24:33.555984 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-5e6daba6-78f8-465f-b82f-7e0916d555c3 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:24:33.556657 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-5e6daba6-78f8-465f-b82f-7e0916d555c3 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:24:33.557478 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-5e6daba6-78f8-465f-b82f-7e0916d555c3 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:24:33.558105 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-5e6daba6-78f8-465f-b82f-7e0916d555c3 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:24:33.558729 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-5e6daba6-78f8-465f-b82f-7e0916d555c3 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:24:33.748446 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-5e6daba6-78f8-465f-b82f-7e0916d555c3 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd945d512ae034ec08838d0a3afde2429', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '880ac4003248441c809a76cfd1b221eb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:24:33.757082 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-5e6daba6-78f8-465f-b82f-7e0916d555c3 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] 10.222.0.52 "GET /compute/v2.1/servers/05bed8e9-f530-418a-9819-4f65dabdcda0" status: 200 len: 1521 microversion: 2.1 time: 0.217844 Aug 27 02:24:33.758038 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 130/264] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:24:33 2022] GET /compute/v2.1/servers/05bed8e9-f530-418a-9819-4f65dabdcda0 => generated 1521 bytes in 219 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:24:34.595774 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-749064f0-d0f2-49a1-a159-bad5e1e03984 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Action: 'create', calling method: .version_select at 0x7f6b1b3b8c10>, body: {"keypair": {"name": "tempest-keypair-105604837", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBP2dRPfl4pk+lGtiM0LBo12cdU2QbEKC+x8onqqoCkQkP8w64RN5N2fTGohc/7NuZYmQaux4Vv5ei8tmAd6QvqnLXdCbNRua6k1Fu2qvX39RVyC4QE214ZRsHUdbHQyBcQ=="}} {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 27 02:24:34.618000 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.quota [None req-749064f0-d0f2-49a1-a159-bad5e1e03984 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Getting quotas for project c938ae5fd22a43e8a01859da2a336b53. Resources: {'key_pairs'} {{(pid=120863) _get_quotas /opt/stack/nova/nova/quota.py:393}} Aug 27 02:24:34.624990 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.quota [None req-749064f0-d0f2-49a1-a159-bad5e1e03984 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Getting quotas for user d750e6e172d34d4a9333e0c059c5442b and project c938ae5fd22a43e8a01859da2a336b53. Resources: {'key_pairs'} {{(pid=120863) _get_quotas /opt/stack/nova/nova/quota.py:383}} Aug 27 02:24:34.685336 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-749064f0-d0f2-49a1-a159-bad5e1e03984 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] 10.222.0.52 "POST /compute/v2.1/os-keypairs" status: 201 len: 400 microversion: 2.20 time: 0.222254 Aug 27 02:24:34.686055 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 135/265] 10.222.0.52 () {62 vars in 1200 bytes} [Sat Aug 27 02:24:34 2022] POST /compute/v2.1/os-keypairs => generated 400 bytes in 224 msecs (HTTP/1.1 201) 9 headers in 364 bytes (2 switches on core 0) Aug 27 02:24:34.781196 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-c37f27c2-3ea7-4c32-bbf4-cbcf3cbed1f2 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Calling method '>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:24:34.785344 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-c37f27c2-3ea7-4c32-bbf4-cbcf3cbed1f2 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] [instance: 05bed8e9-f530-418a-9819-4f65dabdcda0] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:24:34.795280 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-c37f27c2-3ea7-4c32-bbf4-cbcf3cbed1f2 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:24:34.795954 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-c37f27c2-3ea7-4c32-bbf4-cbcf3cbed1f2 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:24:34.796605 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-c37f27c2-3ea7-4c32-bbf4-cbcf3cbed1f2 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:24:34.797986 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-c37f27c2-3ea7-4c32-bbf4-cbcf3cbed1f2 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:24:34.799136 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-c37f27c2-3ea7-4c32-bbf4-cbcf3cbed1f2 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:24:34.799817 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-c37f27c2-3ea7-4c32-bbf4-cbcf3cbed1f2 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:24:34.951967 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-c37f27c2-3ea7-4c32-bbf4-cbcf3cbed1f2 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd945d512ae034ec08838d0a3afde2429', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '880ac4003248441c809a76cfd1b221eb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:24:34.960533 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-c37f27c2-3ea7-4c32-bbf4-cbcf3cbed1f2 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] 10.222.0.52 "GET /compute/v2.1/servers/05bed8e9-f530-418a-9819-4f65dabdcda0" status: 200 len: 1521 microversion: 2.1 time: 0.185867 Aug 27 02:24:34.961111 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 131/266] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:24:34 2022] GET /compute/v2.1/servers/05bed8e9-f530-418a-9819-4f65dabdcda0 => generated 1521 bytes in 187 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:24:35.983509 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-63e8cfd2-c3f9-4f25-ba4d-9452c9e7ce97 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:24:35.984272 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-63e8cfd2-c3f9-4f25-ba4d-9452c9e7ce97 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] [instance: 05bed8e9-f530-418a-9819-4f65dabdcda0] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:24:35.991332 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-63e8cfd2-c3f9-4f25-ba4d-9452c9e7ce97 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:24:35.992033 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-63e8cfd2-c3f9-4f25-ba4d-9452c9e7ce97 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:24:35.992632 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-63e8cfd2-c3f9-4f25-ba4d-9452c9e7ce97 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:24:35.993589 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-63e8cfd2-c3f9-4f25-ba4d-9452c9e7ce97 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:24:35.994227 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-63e8cfd2-c3f9-4f25-ba4d-9452c9e7ce97 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:24:35.994777 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-63e8cfd2-c3f9-4f25-ba4d-9452c9e7ce97 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:24:36.155848 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-63e8cfd2-c3f9-4f25-ba4d-9452c9e7ce97 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd945d512ae034ec08838d0a3afde2429', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '880ac4003248441c809a76cfd1b221eb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:24:36.170112 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-63e8cfd2-c3f9-4f25-ba4d-9452c9e7ce97 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] 10.222.0.52 "GET /compute/v2.1/servers/05bed8e9-f530-418a-9819-4f65dabdcda0" status: 200 len: 1521 microversion: 2.1 time: 0.191067 Aug 27 02:24:36.170693 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 136/267] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:24:35 2022] GET /compute/v2.1/servers/05bed8e9-f530-418a-9819-4f65dabdcda0 => generated 1521 bytes in 192 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:24:36.328981 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-2aae259f-348a-40b6-a114-97ee7b513e4c tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-AttachVolumeShelveTestJSON-server-1020485589", "imageRef": "9fc30b36-b827-46f9-892d-30e808bc2b8a", "flavorRef": "452", "adminPass": "***", "networks": [{"uuid": "daf58feb-9527-44f2-90a3-a19a525b593d"}], "security_groups": [{"name": "tempest-securitygroup--1294230270"}], "key_name": "tempest-keypair-105604837", "user_data": "}} {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 27 02:24:36.555869 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.network.neutron [None req-2aae259f-348a-40b6-a114-97ee7b513e4c tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] validate_networks() for [('daf58feb-9527-44f2-90a3-a19a525b593d', None, None, None, None, None)] {{(pid=120864) validate_networks /opt/stack/nova/nova/network/neutron.py:2626}} Aug 27 02:24:37.164466 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.virt.hardware [None req-2aae259f-348a-40b6-a114-97ee7b513e4c tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Flavor limits 0:0:0 {{(pid=120864) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Aug 27 02:24:37.165136 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.virt.hardware [None req-2aae259f-348a-40b6-a114-97ee7b513e4c tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Image limits 0:0:0 {{(pid=120864) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Aug 27 02:24:37.166399 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.virt.hardware [None req-2aae259f-348a-40b6-a114-97ee7b513e4c tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Flavor pref 0:0:0 {{(pid=120864) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Aug 27 02:24:37.167567 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.virt.hardware [None req-2aae259f-348a-40b6-a114-97ee7b513e4c tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Image pref 0:0:0 {{(pid=120864) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Aug 27 02:24:37.168665 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.virt.hardware [None req-2aae259f-348a-40b6-a114-97ee7b513e4c tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=120864) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Aug 27 02:24:37.182844 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-2aae259f-348a-40b6-a114-97ee7b513e4c tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:24:37.183443 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-2aae259f-348a-40b6-a114-97ee7b513e4c tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:24:37.184136 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-2aae259f-348a-40b6-a114-97ee7b513e4c tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:24:37.185097 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-2aae259f-348a-40b6-a114-97ee7b513e4c tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:24:37.185724 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-2aae259f-348a-40b6-a114-97ee7b513e4c tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:24:37.186273 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-2aae259f-348a-40b6-a114-97ee7b513e4c tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:24:37.195076 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-8b4bf086-a554-46af-9bf2-3c440639c6c1 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:24:37.196568 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-8b4bf086-a554-46af-9bf2-3c440639c6c1 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] [instance: 05bed8e9-f530-418a-9819-4f65dabdcda0] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:24:37.206574 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-8b4bf086-a554-46af-9bf2-3c440639c6c1 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:24:37.207073 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-8b4bf086-a554-46af-9bf2-3c440639c6c1 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:24:37.208213 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-8b4bf086-a554-46af-9bf2-3c440639c6c1 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:24:37.210332 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-8b4bf086-a554-46af-9bf2-3c440639c6c1 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:24:37.211283 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-8b4bf086-a554-46af-9bf2-3c440639c6c1 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:24:37.213834 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-8b4bf086-a554-46af-9bf2-3c440639c6c1 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:24:37.216990 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.quota [None req-2aae259f-348a-40b6-a114-97ee7b513e4c tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Getting quotas for project c938ae5fd22a43e8a01859da2a336b53. Resources: {'cores', 'ram', 'instances'} {{(pid=120864) _get_quotas /opt/stack/nova/nova/quota.py:393}} Aug 27 02:24:37.224810 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.quota [None req-2aae259f-348a-40b6-a114-97ee7b513e4c tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Getting quotas for user d750e6e172d34d4a9333e0c059c5442b and project c938ae5fd22a43e8a01859da2a336b53. Resources: {'cores', 'ram', 'instances'} {{(pid=120864) _get_quotas /opt/stack/nova/nova/quota.py:383}} Aug 27 02:24:37.241662 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-2aae259f-348a-40b6-a114-97ee7b513e4c tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Going to run 1 instances... {{(pid=120864) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Aug 27 02:24:37.244173 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-2aae259f-348a-40b6-a114-97ee7b513e4c tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] [instance: 56dbfa6a-5b6f-4b82-977b-af735e6a6ac9] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='9fc30b36-b827-46f9-892d-30e808bc2b8a',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=120864) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Aug 27 02:24:37.326972 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-2aae259f-348a-40b6-a114-97ee7b513e4c tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] 10.222.0.52 "POST /compute/v2.1/servers" status: 202 len: 402 microversion: 2.20 time: 1.003988 Aug 27 02:24:37.330772 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 132/268] 10.222.0.52 () {62 vars in 1188 bytes} [Sat Aug 27 02:24:36 2022] POST /compute/v2.1/servers => generated 402 bytes in 1009 msecs (HTTP/1.1 202) 10 headers in 453 bytes (1 switches on core 0) Aug 27 02:24:37.349470 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-d07b0051-1e01-4776-a6f6-122e11b5a374 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Calling method '>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:24:37.350861 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-d07b0051-1e01-4776-a6f6-122e11b5a374 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] [instance: 56dbfa6a-5b6f-4b82-977b-af735e6a6ac9] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:24:37.487854 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-d07b0051-1e01-4776-a6f6-122e11b5a374 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:24:37.513210 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-d07b0051-1e01-4776-a6f6-122e11b5a374 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:24:37.514682 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-d07b0051-1e01-4776-a6f6-122e11b5a374 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:24:37.526616 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-d07b0051-1e01-4776-a6f6-122e11b5a374 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] 10.222.0.52 "GET /compute/v2.1/servers/56dbfa6a-5b6f-4b82-977b-af735e6a6ac9" status: 200 len: 1281 microversion: 2.20 time: 0.184602 Aug 27 02:24:37.530191 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 133/269] 10.222.0.52 () {60 vars in 1277 bytes} [Sat Aug 27 02:24:37 2022] GET /compute/v2.1/servers/56dbfa6a-5b6f-4b82-977b-af735e6a6ac9 => generated 1281 bytes in 190 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Aug 27 02:24:37.787230 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-8b4bf086-a554-46af-9bf2-3c440639c6c1 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd945d512ae034ec08838d0a3afde2429', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '880ac4003248441c809a76cfd1b221eb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:24:37.815436 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-8b4bf086-a554-46af-9bf2-3c440639c6c1 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] 10.222.0.52 "GET /compute/v2.1/servers/05bed8e9-f530-418a-9819-4f65dabdcda0" status: 200 len: 1521 microversion: 2.1 time: 0.605139 Aug 27 02:24:37.816741 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 137/270] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:24:37 2022] GET /compute/v2.1/servers/05bed8e9-f530-418a-9819-4f65dabdcda0 => generated 1521 bytes in 627 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 27 02:24:38.552250 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-d2643146-9acf-4a07-beb6-dca7a0f8e87d tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Calling method '>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:24:38.552848 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-d2643146-9acf-4a07-beb6-dca7a0f8e87d tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] [instance: 56dbfa6a-5b6f-4b82-977b-af735e6a6ac9] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:24:38.560038 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-d2643146-9acf-4a07-beb6-dca7a0f8e87d tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:24:38.560530 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-d2643146-9acf-4a07-beb6-dca7a0f8e87d tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:24:38.561051 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-d2643146-9acf-4a07-beb6-dca7a0f8e87d tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:24:38.561749 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-d2643146-9acf-4a07-beb6-dca7a0f8e87d tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:24:38.562236 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-d2643146-9acf-4a07-beb6-dca7a0f8e87d tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:24:38.562660 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-d2643146-9acf-4a07-beb6-dca7a0f8e87d tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:24:38.631859 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-d2643146-9acf-4a07-beb6-dca7a0f8e87d tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:24:38.640409 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-d2643146-9acf-4a07-beb6-dca7a0f8e87d tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:24:38.642096 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-d2643146-9acf-4a07-beb6-dca7a0f8e87d tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:24:38.643192 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-d2643146-9acf-4a07-beb6-dca7a0f8e87d tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] 10.222.0.52 "GET /compute/v2.1/servers/56dbfa6a-5b6f-4b82-977b-af735e6a6ac9" status: 200 len: 1285 microversion: 2.20 time: 0.095169 Aug 27 02:24:38.657437 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 134/271] 10.222.0.52 () {60 vars in 1277 bytes} [Sat Aug 27 02:24:38 2022] GET /compute/v2.1/servers/56dbfa6a-5b6f-4b82-977b-af735e6a6ac9 => generated 1285 bytes in 110 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Aug 27 02:24:38.838647 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-c0b389eb-4bf9-46cc-a76b-a58d691e3f97 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:24:38.839284 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-c0b389eb-4bf9-46cc-a76b-a58d691e3f97 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] [instance: 05bed8e9-f530-418a-9819-4f65dabdcda0] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:24:38.850543 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-c0b389eb-4bf9-46cc-a76b-a58d691e3f97 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:24:38.851213 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-c0b389eb-4bf9-46cc-a76b-a58d691e3f97 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:24:38.851794 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-c0b389eb-4bf9-46cc-a76b-a58d691e3f97 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:24:38.853659 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-c0b389eb-4bf9-46cc-a76b-a58d691e3f97 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:24:38.857109 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-c0b389eb-4bf9-46cc-a76b-a58d691e3f97 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:24:38.861081 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-c0b389eb-4bf9-46cc-a76b-a58d691e3f97 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:24:39.116139 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-c0b389eb-4bf9-46cc-a76b-a58d691e3f97 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd945d512ae034ec08838d0a3afde2429', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '880ac4003248441c809a76cfd1b221eb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:24:39.127799 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-c0b389eb-4bf9-46cc-a76b-a58d691e3f97 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] 10.222.0.52 "GET /compute/v2.1/servers/05bed8e9-f530-418a-9819-4f65dabdcda0" status: 200 len: 1521 microversion: 2.1 time: 0.292749 Aug 27 02:24:39.143339 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 138/272] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:24:38 2022] GET /compute/v2.1/servers/05bed8e9-f530-418a-9819-4f65dabdcda0 => generated 1521 bytes in 308 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:24:39.667113 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-f49c01f5-1268-49c8-9456-be4441baa084 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Calling method '>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:24:39.667684 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-f49c01f5-1268-49c8-9456-be4441baa084 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] [instance: 56dbfa6a-5b6f-4b82-977b-af735e6a6ac9] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:24:39.673949 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-f49c01f5-1268-49c8-9456-be4441baa084 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:24:39.674589 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-f49c01f5-1268-49c8-9456-be4441baa084 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:24:39.675021 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-f49c01f5-1268-49c8-9456-be4441baa084 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:24:39.676038 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-f49c01f5-1268-49c8-9456-be4441baa084 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:24:39.676556 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-f49c01f5-1268-49c8-9456-be4441baa084 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:24:39.677039 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-f49c01f5-1268-49c8-9456-be4441baa084 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:24:39.752421 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-f49c01f5-1268-49c8-9456-be4441baa084 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:24:39.762495 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-f49c01f5-1268-49c8-9456-be4441baa084 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:24:39.764104 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-f49c01f5-1268-49c8-9456-be4441baa084 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:24:39.765386 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-f49c01f5-1268-49c8-9456-be4441baa084 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] 10.222.0.52 "GET /compute/v2.1/servers/56dbfa6a-5b6f-4b82-977b-af735e6a6ac9" status: 200 len: 1285 microversion: 2.20 time: 0.102173 Aug 27 02:24:39.766841 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 135/273] 10.222.0.52 () {60 vars in 1277 bytes} [Sat Aug 27 02:24:39 2022] GET /compute/v2.1/servers/56dbfa6a-5b6f-4b82-977b-af735e6a6ac9 => generated 1285 bytes in 104 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Aug 27 02:24:40.152957 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-3c8d2b64-3cd1-45e6-a12b-dffde1973dfe tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:24:40.154586 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-3c8d2b64-3cd1-45e6-a12b-dffde1973dfe tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] [instance: 05bed8e9-f530-418a-9819-4f65dabdcda0] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:24:40.175149 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-3c8d2b64-3cd1-45e6-a12b-dffde1973dfe tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:24:40.176726 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-3c8d2b64-3cd1-45e6-a12b-dffde1973dfe tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.007s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:24:40.177439 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-3c8d2b64-3cd1-45e6-a12b-dffde1973dfe tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:24:40.178564 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-3c8d2b64-3cd1-45e6-a12b-dffde1973dfe tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:24:40.179258 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-3c8d2b64-3cd1-45e6-a12b-dffde1973dfe tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:24:40.179803 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-3c8d2b64-3cd1-45e6-a12b-dffde1973dfe tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:24:40.478213 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-3c8d2b64-3cd1-45e6-a12b-dffde1973dfe tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd945d512ae034ec08838d0a3afde2429', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '880ac4003248441c809a76cfd1b221eb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:24:40.488560 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-3c8d2b64-3cd1-45e6-a12b-dffde1973dfe tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] 10.222.0.52 "GET /compute/v2.1/servers/05bed8e9-f530-418a-9819-4f65dabdcda0" status: 200 len: 1521 microversion: 2.1 time: 0.340168 Aug 27 02:24:40.489452 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 139/274] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:24:40 2022] GET /compute/v2.1/servers/05bed8e9-f530-418a-9819-4f65dabdcda0 => generated 1521 bytes in 343 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:24:40.789926 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-e61459ea-9ce7-4611-81e3-2eddb4e1f729 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Calling method '>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:24:40.790588 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-e61459ea-9ce7-4611-81e3-2eddb4e1f729 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] [instance: 56dbfa6a-5b6f-4b82-977b-af735e6a6ac9] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:24:40.796947 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-e61459ea-9ce7-4611-81e3-2eddb4e1f729 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:24:40.797725 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-e61459ea-9ce7-4611-81e3-2eddb4e1f729 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:24:40.798322 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-e61459ea-9ce7-4611-81e3-2eddb4e1f729 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:24:40.799218 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-e61459ea-9ce7-4611-81e3-2eddb4e1f729 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:24:40.799831 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-e61459ea-9ce7-4611-81e3-2eddb4e1f729 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:24:40.800402 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-e61459ea-9ce7-4611-81e3-2eddb4e1f729 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:24:40.891357 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-e61459ea-9ce7-4611-81e3-2eddb4e1f729 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:24:40.900906 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-e61459ea-9ce7-4611-81e3-2eddb4e1f729 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:24:40.902556 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-e61459ea-9ce7-4611-81e3-2eddb4e1f729 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:24:40.904169 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-e61459ea-9ce7-4611-81e3-2eddb4e1f729 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] 10.222.0.52 "GET /compute/v2.1/servers/56dbfa6a-5b6f-4b82-977b-af735e6a6ac9" status: 200 len: 1285 microversion: 2.20 time: 0.118386 Aug 27 02:24:40.904906 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 136/275] 10.222.0.52 () {60 vars in 1277 bytes} [Sat Aug 27 02:24:40 2022] GET /compute/v2.1/servers/56dbfa6a-5b6f-4b82-977b-af735e6a6ac9 => generated 1285 bytes in 120 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 27 02:24:41.511832 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-33d19468-d5ef-40de-b8d9-d4f0c878e4b0 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:24:41.512363 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-33d19468-d5ef-40de-b8d9-d4f0c878e4b0 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] [instance: 05bed8e9-f530-418a-9819-4f65dabdcda0] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:24:41.519562 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-33d19468-d5ef-40de-b8d9-d4f0c878e4b0 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:24:41.520387 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-33d19468-d5ef-40de-b8d9-d4f0c878e4b0 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:24:41.521102 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-33d19468-d5ef-40de-b8d9-d4f0c878e4b0 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:24:41.522056 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-33d19468-d5ef-40de-b8d9-d4f0c878e4b0 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:24:41.522614 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-33d19468-d5ef-40de-b8d9-d4f0c878e4b0 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:24:41.523135 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-33d19468-d5ef-40de-b8d9-d4f0c878e4b0 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:24:41.775849 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-33d19468-d5ef-40de-b8d9-d4f0c878e4b0 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd945d512ae034ec08838d0a3afde2429', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '880ac4003248441c809a76cfd1b221eb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:24:41.785806 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-33d19468-d5ef-40de-b8d9-d4f0c878e4b0 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] 10.222.0.52 "GET /compute/v2.1/servers/05bed8e9-f530-418a-9819-4f65dabdcda0" status: 200 len: 1521 microversion: 2.1 time: 0.277956 Aug 27 02:24:41.786634 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 140/276] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:24:41 2022] GET /compute/v2.1/servers/05bed8e9-f530-418a-9819-4f65dabdcda0 => generated 1521 bytes in 280 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:24:41.930808 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-b4264d18-f742-4398-b8a5-4df962369684 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Calling method '>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:24:41.935863 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-b4264d18-f742-4398-b8a5-4df962369684 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] [instance: 56dbfa6a-5b6f-4b82-977b-af735e6a6ac9] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:24:41.942945 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-b4264d18-f742-4398-b8a5-4df962369684 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:24:41.944041 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-b4264d18-f742-4398-b8a5-4df962369684 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:24:41.944926 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-b4264d18-f742-4398-b8a5-4df962369684 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:24:41.946178 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-b4264d18-f742-4398-b8a5-4df962369684 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:24:41.946993 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-b4264d18-f742-4398-b8a5-4df962369684 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:24:41.947612 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-b4264d18-f742-4398-b8a5-4df962369684 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:24:42.059977 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-b4264d18-f742-4398-b8a5-4df962369684 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:24:42.072540 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-b4264d18-f742-4398-b8a5-4df962369684 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:24:42.074621 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-b4264d18-f742-4398-b8a5-4df962369684 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:24:42.076285 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-b4264d18-f742-4398-b8a5-4df962369684 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] 10.222.0.52 "GET /compute/v2.1/servers/56dbfa6a-5b6f-4b82-977b-af735e6a6ac9" status: 200 len: 1285 microversion: 2.20 time: 0.152600 Aug 27 02:24:42.091455 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 137/277] 10.222.0.52 () {60 vars in 1277 bytes} [Sat Aug 27 02:24:41 2022] GET /compute/v2.1/servers/56dbfa6a-5b6f-4b82-977b-af735e6a6ac9 => generated 1285 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 27 02:24:42.807836 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-2f7185cf-243c-4711-b261-06c51e4b6053 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:24:42.809011 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-2f7185cf-243c-4711-b261-06c51e4b6053 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] [instance: 05bed8e9-f530-418a-9819-4f65dabdcda0] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:24:42.815986 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-2f7185cf-243c-4711-b261-06c51e4b6053 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:24:42.817097 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-2f7185cf-243c-4711-b261-06c51e4b6053 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:24:42.817803 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-2f7185cf-243c-4711-b261-06c51e4b6053 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:24:42.818740 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-2f7185cf-243c-4711-b261-06c51e4b6053 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:24:42.819404 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-2f7185cf-243c-4711-b261-06c51e4b6053 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:24:42.820042 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-2f7185cf-243c-4711-b261-06c51e4b6053 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:24:43.050957 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-2f7185cf-243c-4711-b261-06c51e4b6053 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd945d512ae034ec08838d0a3afde2429', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '880ac4003248441c809a76cfd1b221eb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:24:43.063863 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-2f7185cf-243c-4711-b261-06c51e4b6053 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] 10.222.0.52 "GET /compute/v2.1/servers/05bed8e9-f530-418a-9819-4f65dabdcda0" status: 200 len: 1521 microversion: 2.1 time: 0.259747 Aug 27 02:24:43.064722 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 141/278] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:24:42 2022] GET /compute/v2.1/servers/05bed8e9-f530-418a-9819-4f65dabdcda0 => generated 1521 bytes in 261 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:24:43.113843 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-bd37a717-e257-4c9b-a513-88f5c9c205b0 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Calling method '>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:24:43.114550 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-bd37a717-e257-4c9b-a513-88f5c9c205b0 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] [instance: 56dbfa6a-5b6f-4b82-977b-af735e6a6ac9] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:24:43.122697 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-bd37a717-e257-4c9b-a513-88f5c9c205b0 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:24:43.125820 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-bd37a717-e257-4c9b-a513-88f5c9c205b0 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:24:43.131249 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-bd37a717-e257-4c9b-a513-88f5c9c205b0 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:24:43.136269 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-bd37a717-e257-4c9b-a513-88f5c9c205b0 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:24:43.136852 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-bd37a717-e257-4c9b-a513-88f5c9c205b0 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:24:43.137394 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-bd37a717-e257-4c9b-a513-88f5c9c205b0 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:24:43.232286 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-bd37a717-e257-4c9b-a513-88f5c9c205b0 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:24:43.244574 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-bd37a717-e257-4c9b-a513-88f5c9c205b0 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:24:43.246012 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-bd37a717-e257-4c9b-a513-88f5c9c205b0 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:24:43.247117 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-bd37a717-e257-4c9b-a513-88f5c9c205b0 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] 10.222.0.52 "GET /compute/v2.1/servers/56dbfa6a-5b6f-4b82-977b-af735e6a6ac9" status: 200 len: 1333 microversion: 2.20 time: 0.137599 Aug 27 02:24:43.247659 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 138/279] 10.222.0.52 () {60 vars in 1277 bytes} [Sat Aug 27 02:24:43 2022] GET /compute/v2.1/servers/56dbfa6a-5b6f-4b82-977b-af735e6a6ac9 => generated 1333 bytes in 139 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 27 02:24:44.085833 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-195c90c5-70cb-4833-8df0-a409b8a5ce95 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:24:44.091465 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-195c90c5-70cb-4833-8df0-a409b8a5ce95 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] [instance: 05bed8e9-f530-418a-9819-4f65dabdcda0] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:24:44.104435 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-195c90c5-70cb-4833-8df0-a409b8a5ce95 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:24:44.104929 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-195c90c5-70cb-4833-8df0-a409b8a5ce95 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:24:44.105423 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-195c90c5-70cb-4833-8df0-a409b8a5ce95 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:24:44.106130 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-195c90c5-70cb-4833-8df0-a409b8a5ce95 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:24:44.106573 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-195c90c5-70cb-4833-8df0-a409b8a5ce95 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:24:44.107621 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-195c90c5-70cb-4833-8df0-a409b8a5ce95 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:24:44.280969 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-52dd0543-ab06-497c-b9d3-2951c76fa594 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Calling method '>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:24:44.282008 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-52dd0543-ab06-497c-b9d3-2951c76fa594 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] [instance: 56dbfa6a-5b6f-4b82-977b-af735e6a6ac9] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:24:44.298160 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-52dd0543-ab06-497c-b9d3-2951c76fa594 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:24:44.305471 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-52dd0543-ab06-497c-b9d3-2951c76fa594 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.007s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:24:44.306404 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-52dd0543-ab06-497c-b9d3-2951c76fa594 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:24:44.307489 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-52dd0543-ab06-497c-b9d3-2951c76fa594 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:24:44.308128 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-52dd0543-ab06-497c-b9d3-2951c76fa594 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:24:44.308585 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-52dd0543-ab06-497c-b9d3-2951c76fa594 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:24:44.331803 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-195c90c5-70cb-4833-8df0-a409b8a5ce95 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd945d512ae034ec08838d0a3afde2429', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '880ac4003248441c809a76cfd1b221eb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:24:44.357699 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-195c90c5-70cb-4833-8df0-a409b8a5ce95 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] 10.222.0.52 "GET /compute/v2.1/servers/05bed8e9-f530-418a-9819-4f65dabdcda0" status: 200 len: 1501 microversion: 2.1 time: 0.275906 Aug 27 02:24:44.373939 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 142/280] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:24:44 2022] GET /compute/v2.1/servers/05bed8e9-f530-418a-9819-4f65dabdcda0 => generated 1501 bytes in 293 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:24:44.380184 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-2eb6df9e-4fe5-495d-b89d-e1b1754fcea4 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:24:44.380828 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-2eb6df9e-4fe5-495d-b89d-e1b1754fcea4 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] [instance: e2bf7024-0779-4e08-844a-43b78187e770] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:24:44.397054 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-2eb6df9e-4fe5-495d-b89d-e1b1754fcea4 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:24:44.401700 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-2eb6df9e-4fe5-495d-b89d-e1b1754fcea4 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:24:44.402465 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-2eb6df9e-4fe5-495d-b89d-e1b1754fcea4 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:24:44.403458 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-2eb6df9e-4fe5-495d-b89d-e1b1754fcea4 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:24:44.404586 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-2eb6df9e-4fe5-495d-b89d-e1b1754fcea4 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:24:44.405427 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-2eb6df9e-4fe5-495d-b89d-e1b1754fcea4 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:24:44.636067 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG dbcounter [-] [120863] Writing DB stats nova_cell0:SELECT=4 {{(pid=120863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 27 02:24:44.755667 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-52dd0543-ab06-497c-b9d3-2951c76fa594 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:24:44.792139 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-52dd0543-ab06-497c-b9d3-2951c76fa594 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:24:44.796108 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-52dd0543-ab06-497c-b9d3-2951c76fa594 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:24:44.800840 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-52dd0543-ab06-497c-b9d3-2951c76fa594 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] 10.222.0.52 "GET /compute/v2.1/servers/56dbfa6a-5b6f-4b82-977b-af735e6a6ac9" status: 200 len: 1339 microversion: 2.20 time: 0.522642 Aug 27 02:24:44.804063 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 139/281] 10.222.0.52 () {60 vars in 1277 bytes} [Sat Aug 27 02:24:44 2022] GET /compute/v2.1/servers/56dbfa6a-5b6f-4b82-977b-af735e6a6ac9 => generated 1339 bytes in 529 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Aug 27 02:24:44.972771 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-2eb6df9e-4fe5-495d-b89d-e1b1754fcea4 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd945d512ae034ec08838d0a3afde2429', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '880ac4003248441c809a76cfd1b221eb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:24:44.983830 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-2eb6df9e-4fe5-495d-b89d-e1b1754fcea4 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] 10.222.0.52 "GET /compute/v2.1/servers/e2bf7024-0779-4e08-844a-43b78187e770" status: 200 len: 1516 microversion: 2.1 time: 0.608701 Aug 27 02:24:44.984944 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 143/282] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:24:44 2022] GET /compute/v2.1/servers/e2bf7024-0779-4e08-844a-43b78187e770 => generated 1516 bytes in 610 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:24:45.836311 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-efc947b6-357c-4fb5-bdbd-6c37e992d889 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Calling method '>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:24:45.836967 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-efc947b6-357c-4fb5-bdbd-6c37e992d889 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] [instance: 56dbfa6a-5b6f-4b82-977b-af735e6a6ac9] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:24:45.848662 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-efc947b6-357c-4fb5-bdbd-6c37e992d889 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:24:45.849188 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-efc947b6-357c-4fb5-bdbd-6c37e992d889 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:24:45.849624 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-efc947b6-357c-4fb5-bdbd-6c37e992d889 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:24:45.850339 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-efc947b6-357c-4fb5-bdbd-6c37e992d889 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:24:45.850791 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-efc947b6-357c-4fb5-bdbd-6c37e992d889 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:24:45.851205 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-efc947b6-357c-4fb5-bdbd-6c37e992d889 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:24:46.322810 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-efc947b6-357c-4fb5-bdbd-6c37e992d889 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:24:46.335343 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-efc947b6-357c-4fb5-bdbd-6c37e992d889 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:24:46.351234 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-efc947b6-357c-4fb5-bdbd-6c37e992d889 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:24:46.352392 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-efc947b6-357c-4fb5-bdbd-6c37e992d889 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] 10.222.0.52 "GET /compute/v2.1/servers/56dbfa6a-5b6f-4b82-977b-af735e6a6ac9" status: 200 len: 1407 microversion: 2.20 time: 0.521270 Aug 27 02:24:46.365330 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 140/283] 10.222.0.52 () {60 vars in 1277 bytes} [Sat Aug 27 02:24:45 2022] GET /compute/v2.1/servers/56dbfa6a-5b6f-4b82-977b-af735e6a6ac9 => generated 1407 bytes in 535 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 27 02:24:46.724155 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-ad7acf00-686d-4c5d-a138-131360f8162e tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Action: 'action', calling method: >, body: {"rescue": {"adminPass": "***"}} {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 27 02:24:46.727437 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-ad7acf00-686d-4c5d-a138-131360f8162e tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] [instance: e2bf7024-0779-4e08-844a-43b78187e770] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:24:46.747954 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-ad7acf00-686d-4c5d-a138-131360f8162e tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:24:46.748916 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-ad7acf00-686d-4c5d-a138-131360f8162e tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:24:46.749544 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-ad7acf00-686d-4c5d-a138-131360f8162e tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:24:46.750458 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-ad7acf00-686d-4c5d-a138-131360f8162e tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:24:46.751067 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-ad7acf00-686d-4c5d-a138-131360f8162e tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:24:46.751643 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-ad7acf00-686d-4c5d-a138-131360f8162e tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:24:46.803808 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.objects.instance [None req-ad7acf00-686d-4c5d-a138-131360f8162e tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lazy-loading 'flavor' on Instance uuid e2bf7024-0779-4e08-844a-43b78187e770 {{(pid=120863) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Aug 27 02:24:46.953594 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-ad7acf00-686d-4c5d-a138-131360f8162e tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] 10.222.0.52 "POST /compute/v2.1/servers/e2bf7024-0779-4e08-844a-43b78187e770/action" status: 200 len: 32 microversion: 2.1 time: 0.233255 Aug 27 02:24:46.955945 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 144/284] 10.222.0.52 () {60 vars in 1278 bytes} [Sat Aug 27 02:24:46 2022] POST /compute/v2.1/servers/e2bf7024-0779-4e08-844a-43b78187e770/action => generated 32 bytes in 238 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Aug 27 02:24:46.965631 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-945650ba-9b07-48f8-8976-6bd2ee1d17a6 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Calling method '>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:24:46.966398 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-945650ba-9b07-48f8-8976-6bd2ee1d17a6 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] [instance: e2bf7024-0779-4e08-844a-43b78187e770] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:24:46.980928 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-945650ba-9b07-48f8-8976-6bd2ee1d17a6 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:24:46.983396 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-945650ba-9b07-48f8-8976-6bd2ee1d17a6 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:24:46.984389 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-945650ba-9b07-48f8-8976-6bd2ee1d17a6 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:24:46.998422 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-945650ba-9b07-48f8-8976-6bd2ee1d17a6 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:24:46.999830 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-945650ba-9b07-48f8-8976-6bd2ee1d17a6 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:24:47.003658 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-945650ba-9b07-48f8-8976-6bd2ee1d17a6 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:24:47.201378 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG dbcounter [-] [120864] Writing DB stats nova_cell0:SELECT=2 {{(pid=120864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 27 02:24:47.365078 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-945650ba-9b07-48f8-8976-6bd2ee1d17a6 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd945d512ae034ec08838d0a3afde2429', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '880ac4003248441c809a76cfd1b221eb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:24:47.377747 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-945650ba-9b07-48f8-8976-6bd2ee1d17a6 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] 10.222.0.52 "GET /compute/v2.1/servers/e2bf7024-0779-4e08-844a-43b78187e770" status: 200 len: 1522 microversion: 2.1 time: 0.416216 Aug 27 02:24:47.391509 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-c097e883-2f9b-40d4-87ff-fdbd7a8240ad tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:24:47.391881 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 141/285] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:24:46 2022] GET /compute/v2.1/servers/e2bf7024-0779-4e08-844a-43b78187e770 => generated 1522 bytes in 417 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:24:47.391965 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-c097e883-2f9b-40d4-87ff-fdbd7a8240ad tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] [instance: 56dbfa6a-5b6f-4b82-977b-af735e6a6ac9] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:24:47.391965 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-c097e883-2f9b-40d4-87ff-fdbd7a8240ad tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:24:47.391965 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-c097e883-2f9b-40d4-87ff-fdbd7a8240ad tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:24:47.391965 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-c097e883-2f9b-40d4-87ff-fdbd7a8240ad tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:24:47.398635 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-c097e883-2f9b-40d4-87ff-fdbd7a8240ad tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:24:47.402053 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-c097e883-2f9b-40d4-87ff-fdbd7a8240ad tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.008s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:24:47.405098 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-c097e883-2f9b-40d4-87ff-fdbd7a8240ad tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:24:47.756094 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [req-3eaba6df-ade3-4170-83bc-76b7bdded5ec req-55a2c277-7c49-4376-9ff9-913452da37bc service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "56dbfa6a-5b6f-4b82-977b-af735e6a6ac9", "tag": "520bd5fb-723a-4570-83ca-72759c588707"}]} {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 27 02:24:47.794443 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG dbcounter [-] [120864] Writing DB stats nova_cell0:SELECT=8 {{(pid=120864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 27 02:24:47.805541 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [req-3eaba6df-ade3-4170-83bc-76b7bdded5ec req-55a2c277-7c49-4376-9ff9-913452da37bc service nova] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:24:47.806081 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [req-3eaba6df-ade3-4170-83bc-76b7bdded5ec req-55a2c277-7c49-4376-9ff9-913452da37bc service nova] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.009s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:24:47.806510 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [req-3eaba6df-ade3-4170-83bc-76b7bdded5ec req-55a2c277-7c49-4376-9ff9-913452da37bc service nova] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:24:47.854553 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.compute.server_external_events [req-3eaba6df-ade3-4170-83bc-76b7bdded5ec req-55a2c277-7c49-4376-9ff9-913452da37bc service nova] Creating event network-changed:520bd5fb-723a-4570-83ca-72759c588707 for instance 56dbfa6a-5b6f-4b82-977b-af735e6a6ac9 on hv-cinder-84630 Aug 27 02:24:47.861234 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [req-3eaba6df-ade3-4170-83bc-76b7bdded5ec req-55a2c277-7c49-4376-9ff9-913452da37bc service nova] 10.222.0.52 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.109831 Aug 27 02:24:47.862518 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 142/286] 10.222.0.52 () {64 vars in 1317 bytes} [Sat Aug 27 02:24:47 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 112 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Aug 27 02:24:48.056546 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-c097e883-2f9b-40d4-87ff-fdbd7a8240ad tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:24:48.069010 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-c097e883-2f9b-40d4-87ff-fdbd7a8240ad tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:24:48.072031 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-c097e883-2f9b-40d4-87ff-fdbd7a8240ad tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:24:48.073218 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-c097e883-2f9b-40d4-87ff-fdbd7a8240ad tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] 10.222.0.52 "GET /compute/v2.1/servers/56dbfa6a-5b6f-4b82-977b-af735e6a6ac9" status: 200 len: 1407 microversion: 2.20 time: 0.701023 Aug 27 02:24:48.073781 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 145/287] 10.222.0.52 () {60 vars in 1277 bytes} [Sat Aug 27 02:24:47 2022] GET /compute/v2.1/servers/56dbfa6a-5b6f-4b82-977b-af735e6a6ac9 => generated 1407 bytes in 702 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Aug 27 02:24:48.400826 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-c23290ff-c60d-4fa4-a415-10f430241ec2 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Calling method '>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:24:48.401548 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-c23290ff-c60d-4fa4-a415-10f430241ec2 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] [instance: e2bf7024-0779-4e08-844a-43b78187e770] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:24:48.420416 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-c23290ff-c60d-4fa4-a415-10f430241ec2 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:24:48.420893 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-c23290ff-c60d-4fa4-a415-10f430241ec2 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:24:48.423860 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-c23290ff-c60d-4fa4-a415-10f430241ec2 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:24:48.424810 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-c23290ff-c60d-4fa4-a415-10f430241ec2 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:24:48.425304 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-c23290ff-c60d-4fa4-a415-10f430241ec2 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:24:48.425977 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-c23290ff-c60d-4fa4-a415-10f430241ec2 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:24:48.813856 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-c23290ff-c60d-4fa4-a415-10f430241ec2 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd945d512ae034ec08838d0a3afde2429', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '880ac4003248441c809a76cfd1b221eb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:24:48.847479 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-c23290ff-c60d-4fa4-a415-10f430241ec2 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] 10.222.0.52 "GET /compute/v2.1/servers/e2bf7024-0779-4e08-844a-43b78187e770" status: 200 len: 1522 microversion: 2.1 time: 0.451005 Aug 27 02:24:48.856765 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 143/288] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:24:48 2022] GET /compute/v2.1/servers/e2bf7024-0779-4e08-844a-43b78187e770 => generated 1522 bytes in 462 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:24:49.111274 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-3cfe2df9-9bf8-40bb-a663-ce784114f769 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:24:49.122707 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-3cfe2df9-9bf8-40bb-a663-ce784114f769 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] [instance: 56dbfa6a-5b6f-4b82-977b-af735e6a6ac9] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:24:49.148808 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-3cfe2df9-9bf8-40bb-a663-ce784114f769 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:24:49.149763 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-3cfe2df9-9bf8-40bb-a663-ce784114f769 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:24:49.150447 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-3cfe2df9-9bf8-40bb-a663-ce784114f769 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:24:49.151461 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-3cfe2df9-9bf8-40bb-a663-ce784114f769 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:24:49.152424 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-3cfe2df9-9bf8-40bb-a663-ce784114f769 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:24:49.153099 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-3cfe2df9-9bf8-40bb-a663-ce784114f769 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:24:49.360526 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-3cfe2df9-9bf8-40bb-a663-ce784114f769 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:24:49.371092 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-3cfe2df9-9bf8-40bb-a663-ce784114f769 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:24:49.373233 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-3cfe2df9-9bf8-40bb-a663-ce784114f769 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:24:49.374625 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-3cfe2df9-9bf8-40bb-a663-ce784114f769 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] 10.222.0.52 "GET /compute/v2.1/servers/56dbfa6a-5b6f-4b82-977b-af735e6a6ac9" status: 200 len: 1407 microversion: 2.20 time: 0.280974 Aug 27 02:24:49.375419 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 146/289] 10.222.0.52 () {60 vars in 1277 bytes} [Sat Aug 27 02:24:49 2022] GET /compute/v2.1/servers/56dbfa6a-5b6f-4b82-977b-af735e6a6ac9 => generated 1407 bytes in 283 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 27 02:24:49.881105 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-a0b83185-22fb-402d-bd53-1ae6f48279b7 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Calling method '>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:24:49.881725 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-a0b83185-22fb-402d-bd53-1ae6f48279b7 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] [instance: e2bf7024-0779-4e08-844a-43b78187e770] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:24:49.890060 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-a0b83185-22fb-402d-bd53-1ae6f48279b7 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:24:49.890542 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-a0b83185-22fb-402d-bd53-1ae6f48279b7 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:24:49.890972 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-a0b83185-22fb-402d-bd53-1ae6f48279b7 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:24:49.891660 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-a0b83185-22fb-402d-bd53-1ae6f48279b7 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:24:49.892085 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-a0b83185-22fb-402d-bd53-1ae6f48279b7 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:24:49.892500 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-a0b83185-22fb-402d-bd53-1ae6f48279b7 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:24:50.142298 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-a0b83185-22fb-402d-bd53-1ae6f48279b7 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd945d512ae034ec08838d0a3afde2429', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '880ac4003248441c809a76cfd1b221eb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:24:50.154539 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-a0b83185-22fb-402d-bd53-1ae6f48279b7 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] 10.222.0.52 "GET /compute/v2.1/servers/e2bf7024-0779-4e08-844a-43b78187e770" status: 200 len: 1522 microversion: 2.1 time: 0.277410 Aug 27 02:24:50.170378 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 144/290] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:24:49 2022] GET /compute/v2.1/servers/e2bf7024-0779-4e08-844a-43b78187e770 => generated 1522 bytes in 294 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:24:50.399651 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-43635413-792b-4589-b9c7-52b0669682d4 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:24:50.400264 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-43635413-792b-4589-b9c7-52b0669682d4 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] [instance: 56dbfa6a-5b6f-4b82-977b-af735e6a6ac9] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:24:50.407586 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-43635413-792b-4589-b9c7-52b0669682d4 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:24:50.408502 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-43635413-792b-4589-b9c7-52b0669682d4 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:24:50.409283 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-43635413-792b-4589-b9c7-52b0669682d4 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:24:50.410598 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-43635413-792b-4589-b9c7-52b0669682d4 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:24:50.411220 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-43635413-792b-4589-b9c7-52b0669682d4 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:24:50.411821 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-43635413-792b-4589-b9c7-52b0669682d4 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:24:50.618150 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-43635413-792b-4589-b9c7-52b0669682d4 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:24:50.640235 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-43635413-792b-4589-b9c7-52b0669682d4 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:24:50.642395 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-43635413-792b-4589-b9c7-52b0669682d4 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:24:50.644295 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-43635413-792b-4589-b9c7-52b0669682d4 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] 10.222.0.52 "GET /compute/v2.1/servers/56dbfa6a-5b6f-4b82-977b-af735e6a6ac9" status: 200 len: 1407 microversion: 2.20 time: 0.249229 Aug 27 02:24:50.646270 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 147/291] 10.222.0.52 () {60 vars in 1277 bytes} [Sat Aug 27 02:24:50 2022] GET /compute/v2.1/servers/56dbfa6a-5b6f-4b82-977b-af735e6a6ac9 => generated 1407 bytes in 252 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 27 02:24:51.177758 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-c08bd337-a508-46ad-a91d-8f9df62e0e68 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Calling method '>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:24:51.179087 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-c08bd337-a508-46ad-a91d-8f9df62e0e68 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] [instance: e2bf7024-0779-4e08-844a-43b78187e770] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:24:51.192220 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-c08bd337-a508-46ad-a91d-8f9df62e0e68 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:24:51.192709 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-c08bd337-a508-46ad-a91d-8f9df62e0e68 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:24:51.193169 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-c08bd337-a508-46ad-a91d-8f9df62e0e68 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:24:51.194241 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-c08bd337-a508-46ad-a91d-8f9df62e0e68 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:24:51.195026 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-c08bd337-a508-46ad-a91d-8f9df62e0e68 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:24:51.195621 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-c08bd337-a508-46ad-a91d-8f9df62e0e68 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:24:51.395073 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-c08bd337-a508-46ad-a91d-8f9df62e0e68 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd945d512ae034ec08838d0a3afde2429', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '880ac4003248441c809a76cfd1b221eb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:24:51.414621 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-c08bd337-a508-46ad-a91d-8f9df62e0e68 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] 10.222.0.52 "GET /compute/v2.1/servers/e2bf7024-0779-4e08-844a-43b78187e770" status: 200 len: 1522 microversion: 2.1 time: 0.239469 Aug 27 02:24:51.417550 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 145/292] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:24:51 2022] GET /compute/v2.1/servers/e2bf7024-0779-4e08-844a-43b78187e770 => generated 1522 bytes in 246 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:24:51.668377 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-6735f054-6c9a-4dd1-8d03-8fbc9768c17d tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:24:51.668983 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-6735f054-6c9a-4dd1-8d03-8fbc9768c17d tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] [instance: 56dbfa6a-5b6f-4b82-977b-af735e6a6ac9] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:24:51.681713 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-6735f054-6c9a-4dd1-8d03-8fbc9768c17d tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:24:51.683143 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-6735f054-6c9a-4dd1-8d03-8fbc9768c17d tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:24:51.683515 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-6735f054-6c9a-4dd1-8d03-8fbc9768c17d tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:24:51.696303 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-6735f054-6c9a-4dd1-8d03-8fbc9768c17d tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:24:51.696994 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-6735f054-6c9a-4dd1-8d03-8fbc9768c17d tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:24:51.697667 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-6735f054-6c9a-4dd1-8d03-8fbc9768c17d tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:24:51.967431 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-6735f054-6c9a-4dd1-8d03-8fbc9768c17d tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:24:51.976635 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-6735f054-6c9a-4dd1-8d03-8fbc9768c17d tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:24:51.993793 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-6735f054-6c9a-4dd1-8d03-8fbc9768c17d tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:24:51.994965 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-6735f054-6c9a-4dd1-8d03-8fbc9768c17d tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] 10.222.0.52 "GET /compute/v2.1/servers/56dbfa6a-5b6f-4b82-977b-af735e6a6ac9" status: 200 len: 1407 microversion: 2.20 time: 0.330841 Aug 27 02:24:51.995541 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 148/293] 10.222.0.52 () {60 vars in 1277 bytes} [Sat Aug 27 02:24:51 2022] GET /compute/v2.1/servers/56dbfa6a-5b6f-4b82-977b-af735e6a6ac9 => generated 1407 bytes in 332 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 27 02:24:52.438802 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-a49aad0f-04f0-4553-8276-c6dcb5edc9a0 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:24:52.439401 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-a49aad0f-04f0-4553-8276-c6dcb5edc9a0 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] [instance: e2bf7024-0779-4e08-844a-43b78187e770] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:24:52.443272 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-c7a89c46-1571-4e28-971f-4ca019cac642 tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] Action: 'create', calling method: .version_select at 0x7f6b1b38f550>, body: {"keypair": {"name": "tempest-TestShelveInstance-1152661178", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBISlaOvggA5f7VWfZBoCLzVhRoqOCt7m69kMWYBZg4/JNu99V6YrJdsEhvOg/YYNakK02opjS5DZBdzbXNIpYmSCcDNgSy4ZHp3mhOpJjNSG3AJsKF+WMZIA8W6vlABJgQ=="}} {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 27 02:24:52.453735 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-a49aad0f-04f0-4553-8276-c6dcb5edc9a0 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:24:52.454217 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-a49aad0f-04f0-4553-8276-c6dcb5edc9a0 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:24:52.454650 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-a49aad0f-04f0-4553-8276-c6dcb5edc9a0 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:24:52.455344 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-a49aad0f-04f0-4553-8276-c6dcb5edc9a0 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:24:52.460750 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-a49aad0f-04f0-4553-8276-c6dcb5edc9a0 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:24:52.461543 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-a49aad0f-04f0-4553-8276-c6dcb5edc9a0 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:24:52.489731 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.quota [None req-c7a89c46-1571-4e28-971f-4ca019cac642 tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] Getting quotas for project b2ddff853b7546788b645a0344c2297b. Resources: {'key_pairs'} {{(pid=120864) _get_quotas /opt/stack/nova/nova/quota.py:393}} Aug 27 02:24:52.495100 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.quota [None req-c7a89c46-1571-4e28-971f-4ca019cac642 tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] Getting quotas for user 642291c4daa04a6ab3fbce92a8be39c6 and project b2ddff853b7546788b645a0344c2297b. Resources: {'key_pairs'} {{(pid=120864) _get_quotas /opt/stack/nova/nova/quota.py:383}} Aug 27 02:24:52.561003 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-c7a89c46-1571-4e28-971f-4ca019cac642 tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] 10.222.0.52 "POST /compute/v2.1/os-keypairs" status: 200 len: 397 microversion: 2.1 time: 0.460441 Aug 27 02:24:52.568224 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 146/294] 10.222.0.52 () {60 vars in 1159 bytes} [Sat Aug 27 02:24:52 2022] POST /compute/v2.1/os-keypairs => generated 397 bytes in 468 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 27 02:24:52.708504 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-a49aad0f-04f0-4553-8276-c6dcb5edc9a0 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd945d512ae034ec08838d0a3afde2429', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '880ac4003248441c809a76cfd1b221eb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:24:52.731448 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-a49aad0f-04f0-4553-8276-c6dcb5edc9a0 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] 10.222.0.52 "GET /compute/v2.1/servers/e2bf7024-0779-4e08-844a-43b78187e770" status: 200 len: 1522 microversion: 2.1 time: 0.296220 Aug 27 02:24:52.732056 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 149/295] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:24:52 2022] GET /compute/v2.1/servers/e2bf7024-0779-4e08-844a-43b78187e770 => generated 1522 bytes in 297 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:24:53.024288 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-012b3c36-7a0b-411c-806e-5fd270e18783 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Calling method '>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:24:53.025589 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-012b3c36-7a0b-411c-806e-5fd270e18783 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] [instance: 56dbfa6a-5b6f-4b82-977b-af735e6a6ac9] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:24:53.035753 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-012b3c36-7a0b-411c-806e-5fd270e18783 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:24:53.036253 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-012b3c36-7a0b-411c-806e-5fd270e18783 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:24:53.036686 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-012b3c36-7a0b-411c-806e-5fd270e18783 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:24:53.037419 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-012b3c36-7a0b-411c-806e-5fd270e18783 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:24:53.037879 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-012b3c36-7a0b-411c-806e-5fd270e18783 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:24:53.038292 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-012b3c36-7a0b-411c-806e-5fd270e18783 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:24:53.224390 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-012b3c36-7a0b-411c-806e-5fd270e18783 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:24:53.235490 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-012b3c36-7a0b-411c-806e-5fd270e18783 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:24:53.237217 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-012b3c36-7a0b-411c-806e-5fd270e18783 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:24:53.238479 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-012b3c36-7a0b-411c-806e-5fd270e18783 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] 10.222.0.52 "GET /compute/v2.1/servers/56dbfa6a-5b6f-4b82-977b-af735e6a6ac9" status: 200 len: 1407 microversion: 2.20 time: 0.220005 Aug 27 02:24:53.268904 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 147/296] 10.222.0.52 () {60 vars in 1277 bytes} [Sat Aug 27 02:24:53 2022] GET /compute/v2.1/servers/56dbfa6a-5b6f-4b82-977b-af735e6a6ac9 => generated 1407 bytes in 252 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 27 02:24:53.756223 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-5aacb412-9120-47d6-9636-1d82f6db1425 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:24:53.756802 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-5aacb412-9120-47d6-9636-1d82f6db1425 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] [instance: e2bf7024-0779-4e08-844a-43b78187e770] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:24:53.772178 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-5aacb412-9120-47d6-9636-1d82f6db1425 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:24:53.772662 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-5aacb412-9120-47d6-9636-1d82f6db1425 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:24:53.773403 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-5aacb412-9120-47d6-9636-1d82f6db1425 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:24:53.774555 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-5aacb412-9120-47d6-9636-1d82f6db1425 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:24:53.774989 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-5aacb412-9120-47d6-9636-1d82f6db1425 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:24:53.775752 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-5aacb412-9120-47d6-9636-1d82f6db1425 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:24:53.788486 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG dbcounter [-] [120863] Writing DB stats nova_cell1:SELECT=97,nova_cell1:UPDATE=2,nova_cell1:INSERT=1 {{(pid=120863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 27 02:24:53.998287 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-5aacb412-9120-47d6-9636-1d82f6db1425 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd945d512ae034ec08838d0a3afde2429', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '880ac4003248441c809a76cfd1b221eb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:24:54.001455 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-5aacb412-9120-47d6-9636-1d82f6db1425 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] 10.222.0.52 "GET /compute/v2.1/servers/e2bf7024-0779-4e08-844a-43b78187e770" status: 200 len: 1522 microversion: 2.1 time: 0.249033 Aug 27 02:24:54.002279 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 150/297] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:24:53 2022] GET /compute/v2.1/servers/e2bf7024-0779-4e08-844a-43b78187e770 => generated 1522 bytes in 251 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:24:54.136860 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-aeede39a-609f-4087-992c-5360eff911d1 tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-VolumesActionsTest-instance-200413096", "imageRef": "9fc30b36-b827-46f9-892d-30e808bc2b8a", "flavorRef": "452", "networks": [{"uuid": "d622f246-857a-45c7-a4ec-c30529f7e62e"}]}} {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 27 02:24:54.219074 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.network.neutron [None req-aeede39a-609f-4087-992c-5360eff911d1 tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] validate_networks() for [('d622f246-857a-45c7-a4ec-c30529f7e62e', None, None, None, None, None)] {{(pid=120864) validate_networks /opt/stack/nova/nova/network/neutron.py:2626}} Aug 27 02:24:54.269997 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-768e20eb-80fb-4f3d-a755-020620bad0d0 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:24:54.271128 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-768e20eb-80fb-4f3d-a755-020620bad0d0 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] [instance: 56dbfa6a-5b6f-4b82-977b-af735e6a6ac9] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:24:54.273062 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG dbcounter [-] [120863] Writing DB stats nova_api:SELECT=49,nova_api:INSERT=1 {{(pid=120863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 27 02:24:54.283733 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-768e20eb-80fb-4f3d-a755-020620bad0d0 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:24:54.284239 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-768e20eb-80fb-4f3d-a755-020620bad0d0 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:24:54.284672 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-768e20eb-80fb-4f3d-a755-020620bad0d0 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:24:54.291399 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-768e20eb-80fb-4f3d-a755-020620bad0d0 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:24:54.291885 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-768e20eb-80fb-4f3d-a755-020620bad0d0 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:24:54.292326 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-768e20eb-80fb-4f3d-a755-020620bad0d0 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:24:54.614254 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-768e20eb-80fb-4f3d-a755-020620bad0d0 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:24:54.632985 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-768e20eb-80fb-4f3d-a755-020620bad0d0 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:24:54.634406 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-768e20eb-80fb-4f3d-a755-020620bad0d0 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:24:54.636329 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-768e20eb-80fb-4f3d-a755-020620bad0d0 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] 10.222.0.52 "GET /compute/v2.1/servers/56dbfa6a-5b6f-4b82-977b-af735e6a6ac9" status: 200 len: 1407 microversion: 2.20 time: 0.372540 Aug 27 02:24:54.636900 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 151/298] 10.222.0.52 () {60 vars in 1277 bytes} [Sat Aug 27 02:24:54 2022] GET /compute/v2.1/servers/56dbfa6a-5b6f-4b82-977b-af735e6a6ac9 => generated 1407 bytes in 374 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Aug 27 02:24:54.833417 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.virt.hardware [None req-aeede39a-609f-4087-992c-5360eff911d1 tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Flavor limits 0:0:0 {{(pid=120864) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Aug 27 02:24:54.833841 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.virt.hardware [None req-aeede39a-609f-4087-992c-5360eff911d1 tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Image limits 0:0:0 {{(pid=120864) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Aug 27 02:24:54.834315 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.virt.hardware [None req-aeede39a-609f-4087-992c-5360eff911d1 tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Flavor pref 0:0:0 {{(pid=120864) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Aug 27 02:24:54.834719 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.virt.hardware [None req-aeede39a-609f-4087-992c-5360eff911d1 tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Image pref 0:0:0 {{(pid=120864) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Aug 27 02:24:54.835126 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.virt.hardware [None req-aeede39a-609f-4087-992c-5360eff911d1 tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=120864) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Aug 27 02:24:54.845153 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-aeede39a-609f-4087-992c-5360eff911d1 tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:24:54.845962 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-aeede39a-609f-4087-992c-5360eff911d1 tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:24:54.846392 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-aeede39a-609f-4087-992c-5360eff911d1 tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:24:54.847118 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-aeede39a-609f-4087-992c-5360eff911d1 tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:24:54.847695 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-aeede39a-609f-4087-992c-5360eff911d1 tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:24:54.848124 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-aeede39a-609f-4087-992c-5360eff911d1 tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:24:54.868544 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.quota [None req-aeede39a-609f-4087-992c-5360eff911d1 tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Getting quotas for project 7cb3e3496c1a45c8942c3678c0c299e5. Resources: {'cores', 'ram', 'instances'} {{(pid=120864) _get_quotas /opt/stack/nova/nova/quota.py:393}} Aug 27 02:24:54.872990 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.quota [None req-aeede39a-609f-4087-992c-5360eff911d1 tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Getting quotas for user 2e60cf2931f04f0e8e911adea32979dd and project 7cb3e3496c1a45c8942c3678c0c299e5. Resources: {'cores', 'ram', 'instances'} {{(pid=120864) _get_quotas /opt/stack/nova/nova/quota.py:383}} Aug 27 02:24:54.886048 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-aeede39a-609f-4087-992c-5360eff911d1 tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Going to run 1 instances... {{(pid=120864) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Aug 27 02:24:54.888566 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-aeede39a-609f-4087-992c-5360eff911d1 tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] [instance: e49199b7-e998-44fb-9995-e6a13fd44dc6] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='9fc30b36-b827-46f9-892d-30e808bc2b8a',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=120864) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Aug 27 02:24:54.940598 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-aeede39a-609f-4087-992c-5360eff911d1 tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] 10.222.0.52 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 0.808206 Aug 27 02:24:54.953221 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 148/299] 10.222.0.52 () {60 vars in 1147 bytes} [Sat Aug 27 02:24:54 2022] POST /compute/v2.1/servers => generated 380 bytes in 817 msecs (HTTP/1.1 202) 10 headers in 451 bytes (2 switches on core 0) Aug 27 02:24:54.954388 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-45f2301c-e7b8-43c7-98be-703349fbfa76 tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:24:54.954910 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-45f2301c-e7b8-43c7-98be-703349fbfa76 tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] [instance: e49199b7-e998-44fb-9995-e6a13fd44dc6] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:24:55.013850 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-45f2301c-e7b8-43c7-98be-703349fbfa76 tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2e60cf2931f04f0e8e911adea32979dd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7cb3e3496c1a45c8942c3678c0c299e5', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:24:55.022043 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-45f2301c-e7b8-43c7-98be-703349fbfa76 tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] 10.222.0.52 "GET /compute/v2.1/servers/e49199b7-e998-44fb-9995-e6a13fd44dc6" status: 200 len: 1213 microversion: 2.1 time: 0.071479 Aug 27 02:24:55.022973 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 152/300] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:24:54 2022] GET /compute/v2.1/servers/e49199b7-e998-44fb-9995-e6a13fd44dc6 => generated 1213 bytes in 73 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:24:55.024016 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-8eaf536c-98e1-4906-82c9-3074497e7906 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Calling method '>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:24:55.024678 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-8eaf536c-98e1-4906-82c9-3074497e7906 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] [instance: e2bf7024-0779-4e08-844a-43b78187e770] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:24:55.031162 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-8eaf536c-98e1-4906-82c9-3074497e7906 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:24:55.031883 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-8eaf536c-98e1-4906-82c9-3074497e7906 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:24:55.032312 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-8eaf536c-98e1-4906-82c9-3074497e7906 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:24:55.033003 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-8eaf536c-98e1-4906-82c9-3074497e7906 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:24:55.033472 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-8eaf536c-98e1-4906-82c9-3074497e7906 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:24:55.033888 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-8eaf536c-98e1-4906-82c9-3074497e7906 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:24:55.040919 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG dbcounter [-] [120864] Writing DB stats nova_cell1:SELECT=100 {{(pid=120864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 27 02:24:55.200979 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-8eaf536c-98e1-4906-82c9-3074497e7906 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd945d512ae034ec08838d0a3afde2429', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '880ac4003248441c809a76cfd1b221eb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:24:55.211030 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-8eaf536c-98e1-4906-82c9-3074497e7906 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] 10.222.0.52 "GET /compute/v2.1/servers/e2bf7024-0779-4e08-844a-43b78187e770" status: 200 len: 1522 microversion: 2.1 time: 0.191627 Aug 27 02:24:55.211611 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 149/301] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:24:55 2022] GET /compute/v2.1/servers/e2bf7024-0779-4e08-844a-43b78187e770 => generated 1522 bytes in 193 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:24:55.668103 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-627a35b5-2f56-46f8-9932-8fe97eb8368a tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:24:55.668993 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-627a35b5-2f56-46f8-9932-8fe97eb8368a tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] [instance: 56dbfa6a-5b6f-4b82-977b-af735e6a6ac9] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:24:55.676755 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-627a35b5-2f56-46f8-9932-8fe97eb8368a tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:24:55.677391 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-627a35b5-2f56-46f8-9932-8fe97eb8368a tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:24:55.677903 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-627a35b5-2f56-46f8-9932-8fe97eb8368a tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:24:55.678620 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-627a35b5-2f56-46f8-9932-8fe97eb8368a tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:24:55.679063 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-627a35b5-2f56-46f8-9932-8fe97eb8368a tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:24:55.679647 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-627a35b5-2f56-46f8-9932-8fe97eb8368a tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:24:55.827256 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-627a35b5-2f56-46f8-9932-8fe97eb8368a tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:24:55.836385 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-627a35b5-2f56-46f8-9932-8fe97eb8368a tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:24:55.837888 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-627a35b5-2f56-46f8-9932-8fe97eb8368a tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:24:55.839029 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-627a35b5-2f56-46f8-9932-8fe97eb8368a tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] 10.222.0.52 "GET /compute/v2.1/servers/56dbfa6a-5b6f-4b82-977b-af735e6a6ac9" status: 200 len: 1407 microversion: 2.20 time: 0.177272 Aug 27 02:24:55.839582 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 153/302] 10.222.0.52 () {60 vars in 1277 bytes} [Sat Aug 27 02:24:55 2022] GET /compute/v2.1/servers/56dbfa6a-5b6f-4b82-977b-af735e6a6ac9 => generated 1407 bytes in 178 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 27 02:24:56.042806 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-d4574fea-c703-42c5-97f3-edb51a0d41e7 tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Calling method '>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:24:56.043410 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-d4574fea-c703-42c5-97f3-edb51a0d41e7 tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] [instance: e49199b7-e998-44fb-9995-e6a13fd44dc6] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:24:56.050013 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-d4574fea-c703-42c5-97f3-edb51a0d41e7 tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:24:56.050716 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-d4574fea-c703-42c5-97f3-edb51a0d41e7 tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:24:56.051160 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-d4574fea-c703-42c5-97f3-edb51a0d41e7 tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:24:56.051856 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-d4574fea-c703-42c5-97f3-edb51a0d41e7 tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:24:56.052296 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-d4574fea-c703-42c5-97f3-edb51a0d41e7 tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:24:56.052727 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-d4574fea-c703-42c5-97f3-edb51a0d41e7 tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:24:56.118238 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-d4574fea-c703-42c5-97f3-edb51a0d41e7 tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2e60cf2931f04f0e8e911adea32979dd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7cb3e3496c1a45c8942c3678c0c299e5', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:24:56.126236 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-d4574fea-c703-42c5-97f3-edb51a0d41e7 tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] 10.222.0.52 "GET /compute/v2.1/servers/e49199b7-e998-44fb-9995-e6a13fd44dc6" status: 200 len: 1217 microversion: 2.1 time: 0.087272 Aug 27 02:24:56.126803 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 150/303] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:24:56 2022] GET /compute/v2.1/servers/e49199b7-e998-44fb-9995-e6a13fd44dc6 => generated 1217 bytes in 89 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:24:56.234951 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-4a80ef05-a11e-4a7b-a379-7512aafbc11a tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:24:56.235566 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-4a80ef05-a11e-4a7b-a379-7512aafbc11a tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] [instance: e2bf7024-0779-4e08-844a-43b78187e770] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:24:56.243311 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-4a80ef05-a11e-4a7b-a379-7512aafbc11a tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:24:56.244073 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-4a80ef05-a11e-4a7b-a379-7512aafbc11a tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:24:56.244721 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-4a80ef05-a11e-4a7b-a379-7512aafbc11a tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:24:56.245754 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-4a80ef05-a11e-4a7b-a379-7512aafbc11a tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:24:56.246302 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-4a80ef05-a11e-4a7b-a379-7512aafbc11a tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:24:56.246979 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-4a80ef05-a11e-4a7b-a379-7512aafbc11a tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:24:56.392330 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-4a80ef05-a11e-4a7b-a379-7512aafbc11a tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd945d512ae034ec08838d0a3afde2429', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '880ac4003248441c809a76cfd1b221eb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:24:56.404472 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-4a80ef05-a11e-4a7b-a379-7512aafbc11a tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] 10.222.0.52 "GET /compute/v2.1/servers/e2bf7024-0779-4e08-844a-43b78187e770" status: 200 len: 1522 microversion: 2.1 time: 0.173510 Aug 27 02:24:56.408225 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 154/304] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:24:56 2022] GET /compute/v2.1/servers/e2bf7024-0779-4e08-844a-43b78187e770 => generated 1522 bytes in 176 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:24:56.863507 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-bc318ace-248b-44e3-aba6-504b5d884f68 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Calling method '>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:24:56.864095 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-bc318ace-248b-44e3-aba6-504b5d884f68 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] [instance: 56dbfa6a-5b6f-4b82-977b-af735e6a6ac9] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:24:56.870342 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-bc318ace-248b-44e3-aba6-504b5d884f68 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:24:56.870886 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-bc318ace-248b-44e3-aba6-504b5d884f68 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:24:56.871237 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-bc318ace-248b-44e3-aba6-504b5d884f68 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:24:56.871924 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-bc318ace-248b-44e3-aba6-504b5d884f68 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:24:56.872370 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-bc318ace-248b-44e3-aba6-504b5d884f68 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:24:56.872773 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-bc318ace-248b-44e3-aba6-504b5d884f68 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:24:57.014462 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-bc318ace-248b-44e3-aba6-504b5d884f68 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:24:57.026812 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-bc318ace-248b-44e3-aba6-504b5d884f68 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:24:57.029193 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-bc318ace-248b-44e3-aba6-504b5d884f68 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:24:57.031023 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-bc318ace-248b-44e3-aba6-504b5d884f68 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] 10.222.0.52 "GET /compute/v2.1/servers/56dbfa6a-5b6f-4b82-977b-af735e6a6ac9" status: 200 len: 1407 microversion: 2.20 time: 0.172435 Aug 27 02:24:57.031837 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 151/305] 10.222.0.52 () {60 vars in 1277 bytes} [Sat Aug 27 02:24:56 2022] GET /compute/v2.1/servers/56dbfa6a-5b6f-4b82-977b-af735e6a6ac9 => generated 1407 bytes in 174 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 27 02:24:57.149101 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-867021e8-d569-476d-a436-5dfbaa4c565e tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:24:57.149691 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-867021e8-d569-476d-a436-5dfbaa4c565e tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] [instance: e49199b7-e998-44fb-9995-e6a13fd44dc6] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:24:57.156956 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-867021e8-d569-476d-a436-5dfbaa4c565e tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:24:57.157547 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-867021e8-d569-476d-a436-5dfbaa4c565e tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:24:57.158061 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-867021e8-d569-476d-a436-5dfbaa4c565e tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:24:57.158849 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-867021e8-d569-476d-a436-5dfbaa4c565e tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:24:57.159386 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-867021e8-d569-476d-a436-5dfbaa4c565e tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:24:57.160034 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-867021e8-d569-476d-a436-5dfbaa4c565e tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:24:57.233241 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-867021e8-d569-476d-a436-5dfbaa4c565e tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2e60cf2931f04f0e8e911adea32979dd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7cb3e3496c1a45c8942c3678c0c299e5', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:24:57.246254 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-867021e8-d569-476d-a436-5dfbaa4c565e tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] 10.222.0.52 "GET /compute/v2.1/servers/e49199b7-e998-44fb-9995-e6a13fd44dc6" status: 200 len: 1217 microversion: 2.1 time: 0.102059 Aug 27 02:24:57.247276 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 155/306] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:24:57 2022] GET /compute/v2.1/servers/e49199b7-e998-44fb-9995-e6a13fd44dc6 => generated 1217 bytes in 104 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:24:57.436523 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-c4905571-7819-4e16-96d2-86c4d09aff37 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Calling method '>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:24:57.437189 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-c4905571-7819-4e16-96d2-86c4d09aff37 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] [instance: e2bf7024-0779-4e08-844a-43b78187e770] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:24:57.445833 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-c4905571-7819-4e16-96d2-86c4d09aff37 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:24:57.446332 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-c4905571-7819-4e16-96d2-86c4d09aff37 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:24:57.446975 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-c4905571-7819-4e16-96d2-86c4d09aff37 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:24:57.447677 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-c4905571-7819-4e16-96d2-86c4d09aff37 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:24:57.448101 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-c4905571-7819-4e16-96d2-86c4d09aff37 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:24:57.448518 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-c4905571-7819-4e16-96d2-86c4d09aff37 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:24:57.591010 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-c4905571-7819-4e16-96d2-86c4d09aff37 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd945d512ae034ec08838d0a3afde2429', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '880ac4003248441c809a76cfd1b221eb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:24:57.599050 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-c4905571-7819-4e16-96d2-86c4d09aff37 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] 10.222.0.52 "GET /compute/v2.1/servers/e2bf7024-0779-4e08-844a-43b78187e770" status: 200 len: 1522 microversion: 2.1 time: 0.174385 Aug 27 02:24:57.599647 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 152/307] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:24:57 2022] GET /compute/v2.1/servers/e2bf7024-0779-4e08-844a-43b78187e770 => generated 1522 bytes in 176 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:24:58.060269 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-d7c5a3b5-aaa9-49ea-900a-8b4971297940 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:24:58.060866 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-d7c5a3b5-aaa9-49ea-900a-8b4971297940 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] [instance: 56dbfa6a-5b6f-4b82-977b-af735e6a6ac9] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:24:58.068032 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-d7c5a3b5-aaa9-49ea-900a-8b4971297940 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:24:58.068552 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-d7c5a3b5-aaa9-49ea-900a-8b4971297940 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:24:58.068986 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-d7c5a3b5-aaa9-49ea-900a-8b4971297940 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:24:58.069774 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-d7c5a3b5-aaa9-49ea-900a-8b4971297940 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:24:58.070228 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-d7c5a3b5-aaa9-49ea-900a-8b4971297940 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:24:58.070672 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-d7c5a3b5-aaa9-49ea-900a-8b4971297940 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:24:58.213757 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-d7c5a3b5-aaa9-49ea-900a-8b4971297940 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:24:58.222555 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-d7c5a3b5-aaa9-49ea-900a-8b4971297940 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:24:58.224034 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-d7c5a3b5-aaa9-49ea-900a-8b4971297940 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:24:58.225163 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-d7c5a3b5-aaa9-49ea-900a-8b4971297940 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] 10.222.0.52 "GET /compute/v2.1/servers/56dbfa6a-5b6f-4b82-977b-af735e6a6ac9" status: 200 len: 1407 microversion: 2.20 time: 0.168922 Aug 27 02:24:58.225741 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 156/308] 10.222.0.52 () {60 vars in 1277 bytes} [Sat Aug 27 02:24:58 2022] GET /compute/v2.1/servers/56dbfa6a-5b6f-4b82-977b-af735e6a6ac9 => generated 1407 bytes in 170 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 27 02:24:58.270591 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-960fb3e8-5499-493f-b89a-08faca6e30e8 tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Calling method '>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:24:58.271186 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-960fb3e8-5499-493f-b89a-08faca6e30e8 tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] [instance: e49199b7-e998-44fb-9995-e6a13fd44dc6] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:24:58.277314 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-960fb3e8-5499-493f-b89a-08faca6e30e8 tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:24:58.277786 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-960fb3e8-5499-493f-b89a-08faca6e30e8 tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:24:58.278209 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-960fb3e8-5499-493f-b89a-08faca6e30e8 tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:24:58.278884 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-960fb3e8-5499-493f-b89a-08faca6e30e8 tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:24:58.279322 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-960fb3e8-5499-493f-b89a-08faca6e30e8 tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:24:58.279731 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-960fb3e8-5499-493f-b89a-08faca6e30e8 tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:24:58.343314 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-960fb3e8-5499-493f-b89a-08faca6e30e8 tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2e60cf2931f04f0e8e911adea32979dd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7cb3e3496c1a45c8942c3678c0c299e5', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:24:58.352169 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-960fb3e8-5499-493f-b89a-08faca6e30e8 tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] 10.222.0.52 "GET /compute/v2.1/servers/e49199b7-e998-44fb-9995-e6a13fd44dc6" status: 200 len: 1217 microversion: 2.1 time: 0.086094 Aug 27 02:24:58.352807 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 153/309] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:24:58 2022] GET /compute/v2.1/servers/e49199b7-e998-44fb-9995-e6a13fd44dc6 => generated 1217 bytes in 87 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:24:58.625925 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-e3406553-0d68-463c-8906-349e52f025cf tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:24:58.626650 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-e3406553-0d68-463c-8906-349e52f025cf tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] [instance: e2bf7024-0779-4e08-844a-43b78187e770] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:24:58.633135 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-e3406553-0d68-463c-8906-349e52f025cf tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:24:58.633634 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-e3406553-0d68-463c-8906-349e52f025cf tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:24:58.634071 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-e3406553-0d68-463c-8906-349e52f025cf tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:24:58.634764 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-e3406553-0d68-463c-8906-349e52f025cf tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:24:58.635215 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-e3406553-0d68-463c-8906-349e52f025cf tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:24:58.635639 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-e3406553-0d68-463c-8906-349e52f025cf tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:24:58.782807 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-e3406553-0d68-463c-8906-349e52f025cf tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd945d512ae034ec08838d0a3afde2429', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '880ac4003248441c809a76cfd1b221eb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:24:58.791163 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-e3406553-0d68-463c-8906-349e52f025cf tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] 10.222.0.52 "GET /compute/v2.1/servers/e2bf7024-0779-4e08-844a-43b78187e770" status: 200 len: 1522 microversion: 2.1 time: 0.169364 Aug 27 02:24:58.791765 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 157/310] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:24:58 2022] GET /compute/v2.1/servers/e2bf7024-0779-4e08-844a-43b78187e770 => generated 1522 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:24:59.251310 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-4be59e42-a976-4db4-abd7-5c2db4fa6888 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Calling method '>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:24:59.252104 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-4be59e42-a976-4db4-abd7-5c2db4fa6888 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] [instance: 56dbfa6a-5b6f-4b82-977b-af735e6a6ac9] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:24:59.261817 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-4be59e42-a976-4db4-abd7-5c2db4fa6888 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:24:59.262434 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-4be59e42-a976-4db4-abd7-5c2db4fa6888 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:24:59.262975 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-4be59e42-a976-4db4-abd7-5c2db4fa6888 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:24:59.263986 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-4be59e42-a976-4db4-abd7-5c2db4fa6888 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:24:59.264619 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-4be59e42-a976-4db4-abd7-5c2db4fa6888 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:24:59.265194 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-4be59e42-a976-4db4-abd7-5c2db4fa6888 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:24:59.373006 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-c56771a9-0aa8-4cf8-a2a8-1d82ed8a74c3 tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:24:59.374254 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-c56771a9-0aa8-4cf8-a2a8-1d82ed8a74c3 tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] [instance: e49199b7-e998-44fb-9995-e6a13fd44dc6] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:24:59.380667 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-c56771a9-0aa8-4cf8-a2a8-1d82ed8a74c3 tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:24:59.381153 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-c56771a9-0aa8-4cf8-a2a8-1d82ed8a74c3 tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:24:59.381606 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-c56771a9-0aa8-4cf8-a2a8-1d82ed8a74c3 tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:24:59.382289 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-c56771a9-0aa8-4cf8-a2a8-1d82ed8a74c3 tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:24:59.382734 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-c56771a9-0aa8-4cf8-a2a8-1d82ed8a74c3 tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:24:59.383150 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-c56771a9-0aa8-4cf8-a2a8-1d82ed8a74c3 tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:24:59.407765 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-4be59e42-a976-4db4-abd7-5c2db4fa6888 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:24:59.416627 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-4be59e42-a976-4db4-abd7-5c2db4fa6888 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:24:59.418076 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-4be59e42-a976-4db4-abd7-5c2db4fa6888 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:24:59.419186 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-4be59e42-a976-4db4-abd7-5c2db4fa6888 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] 10.222.0.52 "GET /compute/v2.1/servers/56dbfa6a-5b6f-4b82-977b-af735e6a6ac9" status: 200 len: 1407 microversion: 2.20 time: 0.174525 Aug 27 02:24:59.419763 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 154/311] 10.222.0.52 () {60 vars in 1277 bytes} [Sat Aug 27 02:24:59 2022] GET /compute/v2.1/servers/56dbfa6a-5b6f-4b82-977b-af735e6a6ac9 => generated 1407 bytes in 176 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 27 02:24:59.451228 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-c56771a9-0aa8-4cf8-a2a8-1d82ed8a74c3 tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2e60cf2931f04f0e8e911adea32979dd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7cb3e3496c1a45c8942c3678c0c299e5', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:24:59.459405 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-c56771a9-0aa8-4cf8-a2a8-1d82ed8a74c3 tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] 10.222.0.52 "GET /compute/v2.1/servers/e49199b7-e998-44fb-9995-e6a13fd44dc6" status: 200 len: 1217 microversion: 2.1 time: 0.090191 Aug 27 02:24:59.460402 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 158/312] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:24:59 2022] GET /compute/v2.1/servers/e49199b7-e998-44fb-9995-e6a13fd44dc6 => generated 1217 bytes in 92 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:24:59.815447 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-3cb9c839-c73b-4086-859e-790795a3d730 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Calling method '>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:24:59.816991 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-3cb9c839-c73b-4086-859e-790795a3d730 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] [instance: e2bf7024-0779-4e08-844a-43b78187e770] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:24:59.831011 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-3cb9c839-c73b-4086-859e-790795a3d730 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:24:59.831537 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-3cb9c839-c73b-4086-859e-790795a3d730 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:24:59.832114 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-3cb9c839-c73b-4086-859e-790795a3d730 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:24:59.833246 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-3cb9c839-c73b-4086-859e-790795a3d730 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:24:59.833780 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-3cb9c839-c73b-4086-859e-790795a3d730 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:24:59.834428 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-3cb9c839-c73b-4086-859e-790795a3d730 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:24:59.986267 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-3cb9c839-c73b-4086-859e-790795a3d730 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd945d512ae034ec08838d0a3afde2429', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '880ac4003248441c809a76cfd1b221eb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:24:59.994735 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-3cb9c839-c73b-4086-859e-790795a3d730 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] 10.222.0.52 "GET /compute/v2.1/servers/e2bf7024-0779-4e08-844a-43b78187e770" status: 200 len: 1522 microversion: 2.1 time: 0.186477 Aug 27 02:24:59.995311 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 155/313] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:24:59 2022] GET /compute/v2.1/servers/e2bf7024-0779-4e08-844a-43b78187e770 => generated 1522 bytes in 188 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:25:00.447897 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-5d5b11d7-5667-4103-a5d7-f4d706e2a4f2 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:25:00.448487 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-5d5b11d7-5667-4103-a5d7-f4d706e2a4f2 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] [instance: 56dbfa6a-5b6f-4b82-977b-af735e6a6ac9] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:25:00.455566 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-5d5b11d7-5667-4103-a5d7-f4d706e2a4f2 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:25:00.456047 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-5d5b11d7-5667-4103-a5d7-f4d706e2a4f2 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:25:00.456913 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-5d5b11d7-5667-4103-a5d7-f4d706e2a4f2 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:25:00.458991 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-5d5b11d7-5667-4103-a5d7-f4d706e2a4f2 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:25:00.459860 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-5d5b11d7-5667-4103-a5d7-f4d706e2a4f2 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:25:00.462621 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-5d5b11d7-5667-4103-a5d7-f4d706e2a4f2 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:25:00.483517 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-ac12d963-bb6f-4955-b29e-92b0680346db tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Calling method '>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:25:00.484317 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-ac12d963-bb6f-4955-b29e-92b0680346db tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] [instance: e49199b7-e998-44fb-9995-e6a13fd44dc6] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:25:00.492530 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-ac12d963-bb6f-4955-b29e-92b0680346db tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:25:00.493483 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-ac12d963-bb6f-4955-b29e-92b0680346db tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:25:00.494072 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-ac12d963-bb6f-4955-b29e-92b0680346db tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:25:00.495083 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-ac12d963-bb6f-4955-b29e-92b0680346db tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:25:00.495722 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-ac12d963-bb6f-4955-b29e-92b0680346db tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:25:00.496292 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-ac12d963-bb6f-4955-b29e-92b0680346db tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:25:00.581956 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-ac12d963-bb6f-4955-b29e-92b0680346db tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2e60cf2931f04f0e8e911adea32979dd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7cb3e3496c1a45c8942c3678c0c299e5', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:25:00.591477 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-ac12d963-bb6f-4955-b29e-92b0680346db tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] 10.222.0.52 "GET /compute/v2.1/servers/e49199b7-e998-44fb-9995-e6a13fd44dc6" status: 200 len: 1265 microversion: 2.1 time: 0.111695 Aug 27 02:25:00.604921 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 156/314] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:25:00 2022] GET /compute/v2.1/servers/e49199b7-e998-44fb-9995-e6a13fd44dc6 => generated 1265 bytes in 113 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:25:00.622582 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-5d5b11d7-5667-4103-a5d7-f4d706e2a4f2 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:25:00.631470 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-5d5b11d7-5667-4103-a5d7-f4d706e2a4f2 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:25:00.632885 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-5d5b11d7-5667-4103-a5d7-f4d706e2a4f2 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:25:00.634023 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-5d5b11d7-5667-4103-a5d7-f4d706e2a4f2 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] 10.222.0.52 "GET /compute/v2.1/servers/56dbfa6a-5b6f-4b82-977b-af735e6a6ac9" status: 200 len: 1407 microversion: 2.20 time: 0.190271 Aug 27 02:25:00.634607 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 159/315] 10.222.0.52 () {60 vars in 1277 bytes} [Sat Aug 27 02:25:00 2022] GET /compute/v2.1/servers/56dbfa6a-5b6f-4b82-977b-af735e6a6ac9 => generated 1407 bytes in 191 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 27 02:25:01.023772 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-33960dd9-22f6-4a1d-a78b-72d9a00ef03f tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Calling method '>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:25:01.024278 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-33960dd9-22f6-4a1d-a78b-72d9a00ef03f tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] [instance: e2bf7024-0779-4e08-844a-43b78187e770] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:25:01.030859 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-33960dd9-22f6-4a1d-a78b-72d9a00ef03f tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:25:01.031392 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-33960dd9-22f6-4a1d-a78b-72d9a00ef03f tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:25:01.031887 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-33960dd9-22f6-4a1d-a78b-72d9a00ef03f tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:25:01.032706 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-33960dd9-22f6-4a1d-a78b-72d9a00ef03f tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:25:01.033295 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-33960dd9-22f6-4a1d-a78b-72d9a00ef03f tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:25:01.033813 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-33960dd9-22f6-4a1d-a78b-72d9a00ef03f tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:25:01.170223 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-33960dd9-22f6-4a1d-a78b-72d9a00ef03f tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd945d512ae034ec08838d0a3afde2429', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '880ac4003248441c809a76cfd1b221eb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:25:01.177900 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-33960dd9-22f6-4a1d-a78b-72d9a00ef03f tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] 10.222.0.52 "GET /compute/v2.1/servers/e2bf7024-0779-4e08-844a-43b78187e770" status: 200 len: 1522 microversion: 2.1 time: 0.158651 Aug 27 02:25:01.178459 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 157/316] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:25:01 2022] GET /compute/v2.1/servers/e2bf7024-0779-4e08-844a-43b78187e770 => generated 1522 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:25:01.618861 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-6698d913-dc69-422d-89d8-9f907e137303 tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:25:01.619845 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-6698d913-dc69-422d-89d8-9f907e137303 tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] [instance: e49199b7-e998-44fb-9995-e6a13fd44dc6] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:25:01.634028 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-6698d913-dc69-422d-89d8-9f907e137303 tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:25:01.634684 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-6698d913-dc69-422d-89d8-9f907e137303 tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:25:01.635204 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-6698d913-dc69-422d-89d8-9f907e137303 tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:25:01.635982 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-6698d913-dc69-422d-89d8-9f907e137303 tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:25:01.636596 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-6698d913-dc69-422d-89d8-9f907e137303 tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:25:01.637186 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-6698d913-dc69-422d-89d8-9f907e137303 tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:25:01.658578 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-efbd3e0f-ae10-4851-a835-126aa201f341 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Calling method '>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:25:01.659348 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-efbd3e0f-ae10-4851-a835-126aa201f341 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] [instance: 56dbfa6a-5b6f-4b82-977b-af735e6a6ac9] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:25:01.661716 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG dbcounter [-] [120864] Writing DB stats nova_api:SELECT=93,nova_api:INSERT=7 {{(pid=120864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 27 02:25:01.666906 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-efbd3e0f-ae10-4851-a835-126aa201f341 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:25:01.667406 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-efbd3e0f-ae10-4851-a835-126aa201f341 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:25:01.667829 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-efbd3e0f-ae10-4851-a835-126aa201f341 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:25:01.668635 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-efbd3e0f-ae10-4851-a835-126aa201f341 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:25:01.669160 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-efbd3e0f-ae10-4851-a835-126aa201f341 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:25:01.669594 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-efbd3e0f-ae10-4851-a835-126aa201f341 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:25:01.708709 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-6698d913-dc69-422d-89d8-9f907e137303 tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2e60cf2931f04f0e8e911adea32979dd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7cb3e3496c1a45c8942c3678c0c299e5', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:25:01.717300 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-6698d913-dc69-422d-89d8-9f907e137303 tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] 10.222.0.52 "GET /compute/v2.1/servers/e49199b7-e998-44fb-9995-e6a13fd44dc6" status: 200 len: 1265 microversion: 2.1 time: 0.106463 Aug 27 02:25:01.717915 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 160/317] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:25:01 2022] GET /compute/v2.1/servers/e49199b7-e998-44fb-9995-e6a13fd44dc6 => generated 1265 bytes in 108 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:25:01.808291 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-efbd3e0f-ae10-4851-a835-126aa201f341 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:25:01.817414 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-efbd3e0f-ae10-4851-a835-126aa201f341 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:25:01.818841 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-efbd3e0f-ae10-4851-a835-126aa201f341 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:25:01.819926 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-efbd3e0f-ae10-4851-a835-126aa201f341 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] 10.222.0.52 "GET /compute/v2.1/servers/56dbfa6a-5b6f-4b82-977b-af735e6a6ac9" status: 200 len: 1407 microversion: 2.20 time: 0.165433 Aug 27 02:25:01.820473 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 158/318] 10.222.0.52 () {60 vars in 1277 bytes} [Sat Aug 27 02:25:01 2022] GET /compute/v2.1/servers/56dbfa6a-5b6f-4b82-977b-af735e6a6ac9 => generated 1407 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 27 02:25:02.199173 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-461c5708-5013-427e-a5db-94638883ce3e tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:25:02.200268 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-461c5708-5013-427e-a5db-94638883ce3e tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] [instance: e2bf7024-0779-4e08-844a-43b78187e770] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:25:02.206339 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-461c5708-5013-427e-a5db-94638883ce3e tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:25:02.207459 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-461c5708-5013-427e-a5db-94638883ce3e tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:25:02.207800 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-461c5708-5013-427e-a5db-94638883ce3e tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:25:02.208532 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-461c5708-5013-427e-a5db-94638883ce3e tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:25:02.208983 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-461c5708-5013-427e-a5db-94638883ce3e tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:25:02.209741 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-461c5708-5013-427e-a5db-94638883ce3e tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:25:02.354000 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-461c5708-5013-427e-a5db-94638883ce3e tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd945d512ae034ec08838d0a3afde2429', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '880ac4003248441c809a76cfd1b221eb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:25:02.362043 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-461c5708-5013-427e-a5db-94638883ce3e tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] 10.222.0.52 "GET /compute/v2.1/servers/e2bf7024-0779-4e08-844a-43b78187e770" status: 200 len: 1522 microversion: 2.1 time: 0.167177 Aug 27 02:25:02.362606 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 161/319] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:25:02 2022] GET /compute/v2.1/servers/e2bf7024-0779-4e08-844a-43b78187e770 => generated 1522 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:25:02.739928 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-daa96e90-86d0-484e-b23c-f4ed1eb6ded4 tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Calling method '>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:25:02.740529 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-daa96e90-86d0-484e-b23c-f4ed1eb6ded4 tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] [instance: e49199b7-e998-44fb-9995-e6a13fd44dc6] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:25:02.746672 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-daa96e90-86d0-484e-b23c-f4ed1eb6ded4 tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:25:02.747176 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-daa96e90-86d0-484e-b23c-f4ed1eb6ded4 tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:25:02.747607 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-daa96e90-86d0-484e-b23c-f4ed1eb6ded4 tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:25:02.748269 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-daa96e90-86d0-484e-b23c-f4ed1eb6ded4 tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:25:02.748721 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-daa96e90-86d0-484e-b23c-f4ed1eb6ded4 tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:25:02.749179 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-daa96e90-86d0-484e-b23c-f4ed1eb6ded4 tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:25:02.814099 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-daa96e90-86d0-484e-b23c-f4ed1eb6ded4 tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2e60cf2931f04f0e8e911adea32979dd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7cb3e3496c1a45c8942c3678c0c299e5', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:25:02.825472 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-daa96e90-86d0-484e-b23c-f4ed1eb6ded4 tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] 10.222.0.52 "GET /compute/v2.1/servers/e49199b7-e998-44fb-9995-e6a13fd44dc6" status: 200 len: 1265 microversion: 2.1 time: 0.090330 Aug 27 02:25:02.826158 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 159/320] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:25:02 2022] GET /compute/v2.1/servers/e49199b7-e998-44fb-9995-e6a13fd44dc6 => generated 1265 bytes in 92 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:25:02.843071 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-15c0020e-818f-40b7-a759-98275009ca64 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:25:02.843694 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-15c0020e-818f-40b7-a759-98275009ca64 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] [instance: 56dbfa6a-5b6f-4b82-977b-af735e6a6ac9] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:25:02.851036 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-15c0020e-818f-40b7-a759-98275009ca64 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:25:02.851546 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-15c0020e-818f-40b7-a759-98275009ca64 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:25:02.852081 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-15c0020e-818f-40b7-a759-98275009ca64 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:25:02.853174 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-15c0020e-818f-40b7-a759-98275009ca64 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:25:02.853691 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-15c0020e-818f-40b7-a759-98275009ca64 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:25:02.854271 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-15c0020e-818f-40b7-a759-98275009ca64 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:25:02.992270 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-15c0020e-818f-40b7-a759-98275009ca64 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:25:03.001159 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-15c0020e-818f-40b7-a759-98275009ca64 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:25:03.002676 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-15c0020e-818f-40b7-a759-98275009ca64 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:25:03.003795 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-15c0020e-818f-40b7-a759-98275009ca64 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] 10.222.0.52 "GET /compute/v2.1/servers/56dbfa6a-5b6f-4b82-977b-af735e6a6ac9" status: 200 len: 1407 microversion: 2.20 time: 0.164816 Aug 27 02:25:03.004438 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 162/321] 10.222.0.52 () {60 vars in 1277 bytes} [Sat Aug 27 02:25:02 2022] GET /compute/v2.1/servers/56dbfa6a-5b6f-4b82-977b-af735e6a6ac9 => generated 1407 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 27 02:25:03.391575 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-e4a76c49-ad3b-4803-9779-9626a5d545c3 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Calling method '>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:25:03.392394 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-e4a76c49-ad3b-4803-9779-9626a5d545c3 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] [instance: e2bf7024-0779-4e08-844a-43b78187e770] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:25:03.402188 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-e4a76c49-ad3b-4803-9779-9626a5d545c3 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:25:03.402823 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-e4a76c49-ad3b-4803-9779-9626a5d545c3 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:25:03.403495 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-e4a76c49-ad3b-4803-9779-9626a5d545c3 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:25:03.403495 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-e4a76c49-ad3b-4803-9779-9626a5d545c3 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:25:03.404497 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-e4a76c49-ad3b-4803-9779-9626a5d545c3 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:25:03.405068 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-e4a76c49-ad3b-4803-9779-9626a5d545c3 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:25:03.551087 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-e4a76c49-ad3b-4803-9779-9626a5d545c3 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd945d512ae034ec08838d0a3afde2429', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '880ac4003248441c809a76cfd1b221eb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:25:03.559508 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-e4a76c49-ad3b-4803-9779-9626a5d545c3 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] 10.222.0.52 "GET /compute/v2.1/servers/e2bf7024-0779-4e08-844a-43b78187e770" status: 200 len: 1522 microversion: 2.1 time: 0.173526 Aug 27 02:25:03.560121 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 160/322] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:25:03 2022] GET /compute/v2.1/servers/e2bf7024-0779-4e08-844a-43b78187e770 => generated 1522 bytes in 175 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:25:03.846749 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-8c225555-6928-414c-adcc-f7f4db208cb3 tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:25:03.847470 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-8c225555-6928-414c-adcc-f7f4db208cb3 tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] [instance: e49199b7-e998-44fb-9995-e6a13fd44dc6] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:25:03.854757 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-8c225555-6928-414c-adcc-f7f4db208cb3 tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:25:03.855242 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-8c225555-6928-414c-adcc-f7f4db208cb3 tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:25:03.855675 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-8c225555-6928-414c-adcc-f7f4db208cb3 tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:25:03.856342 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-8c225555-6928-414c-adcc-f7f4db208cb3 tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:25:03.856816 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-8c225555-6928-414c-adcc-f7f4db208cb3 tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:25:03.857275 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-8c225555-6928-414c-adcc-f7f4db208cb3 tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:25:03.928747 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-8c225555-6928-414c-adcc-f7f4db208cb3 tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2e60cf2931f04f0e8e911adea32979dd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7cb3e3496c1a45c8942c3678c0c299e5', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:25:03.937331 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-8c225555-6928-414c-adcc-f7f4db208cb3 tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] 10.222.0.52 "GET /compute/v2.1/servers/e49199b7-e998-44fb-9995-e6a13fd44dc6" status: 200 len: 1265 microversion: 2.1 time: 0.094300 Aug 27 02:25:03.937947 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 163/323] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:25:03 2022] GET /compute/v2.1/servers/e49199b7-e998-44fb-9995-e6a13fd44dc6 => generated 1265 bytes in 96 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:25:04.029617 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-95346c45-3895-4a0a-aca6-fe81ff1f5b15 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Calling method '>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:25:04.030223 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-95346c45-3895-4a0a-aca6-fe81ff1f5b15 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] [instance: 56dbfa6a-5b6f-4b82-977b-af735e6a6ac9] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:25:04.038064 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-95346c45-3895-4a0a-aca6-fe81ff1f5b15 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:25:04.038551 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-95346c45-3895-4a0a-aca6-fe81ff1f5b15 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:25:04.038986 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-95346c45-3895-4a0a-aca6-fe81ff1f5b15 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:25:04.039681 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-95346c45-3895-4a0a-aca6-fe81ff1f5b15 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:25:04.040129 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-95346c45-3895-4a0a-aca6-fe81ff1f5b15 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:25:04.040547 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-95346c45-3895-4a0a-aca6-fe81ff1f5b15 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:25:04.177079 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-95346c45-3895-4a0a-aca6-fe81ff1f5b15 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:25:04.185425 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-95346c45-3895-4a0a-aca6-fe81ff1f5b15 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:25:04.186843 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-95346c45-3895-4a0a-aca6-fe81ff1f5b15 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:25:04.187926 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-95346c45-3895-4a0a-aca6-fe81ff1f5b15 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] 10.222.0.52 "GET /compute/v2.1/servers/56dbfa6a-5b6f-4b82-977b-af735e6a6ac9" status: 200 len: 1407 microversion: 2.20 time: 0.163976 Aug 27 02:25:04.188470 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 161/324] 10.222.0.52 () {60 vars in 1277 bytes} [Sat Aug 27 02:25:04 2022] GET /compute/v2.1/servers/56dbfa6a-5b6f-4b82-977b-af735e6a6ac9 => generated 1407 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 27 02:25:04.590943 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-346da790-593e-4591-9d40-730be601b317 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:25:04.591801 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-346da790-593e-4591-9d40-730be601b317 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] [instance: e2bf7024-0779-4e08-844a-43b78187e770] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:25:04.599629 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-346da790-593e-4591-9d40-730be601b317 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:25:04.600114 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-346da790-593e-4591-9d40-730be601b317 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:25:04.600549 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-346da790-593e-4591-9d40-730be601b317 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:25:04.601264 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-346da790-593e-4591-9d40-730be601b317 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:25:04.601720 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-346da790-593e-4591-9d40-730be601b317 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:25:04.602145 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-346da790-593e-4591-9d40-730be601b317 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:25:04.736852 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-346da790-593e-4591-9d40-730be601b317 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd945d512ae034ec08838d0a3afde2429', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '880ac4003248441c809a76cfd1b221eb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:25:04.745269 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-346da790-593e-4591-9d40-730be601b317 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] 10.222.0.52 "GET /compute/v2.1/servers/e2bf7024-0779-4e08-844a-43b78187e770" status: 200 len: 1522 microversion: 2.1 time: 0.160262 Aug 27 02:25:04.745828 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 164/325] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:25:04 2022] GET /compute/v2.1/servers/e2bf7024-0779-4e08-844a-43b78187e770 => generated 1522 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:25:04.964349 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-15d4fea5-be5e-4fec-ab2b-968a361ffb56 tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Calling method '>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:25:04.965210 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-15d4fea5-be5e-4fec-ab2b-968a361ffb56 tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] [instance: e49199b7-e998-44fb-9995-e6a13fd44dc6] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:25:04.968148 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG dbcounter [-] [120864] Writing DB stats nova_cell0:SELECT=2 {{(pid=120864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 27 02:25:04.969179 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG dbcounter [-] [120864] Writing DB stats nova_cell0:SELECT=10 {{(pid=120864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 27 02:25:04.980721 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-15d4fea5-be5e-4fec-ab2b-968a361ffb56 tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:25:04.981837 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-15d4fea5-be5e-4fec-ab2b-968a361ffb56 tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:25:04.982348 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-15d4fea5-be5e-4fec-ab2b-968a361ffb56 tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:25:04.983283 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-15d4fea5-be5e-4fec-ab2b-968a361ffb56 tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:25:04.983942 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-15d4fea5-be5e-4fec-ab2b-968a361ffb56 tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:25:04.984832 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-15d4fea5-be5e-4fec-ab2b-968a361ffb56 tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:25:05.061283 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-15d4fea5-be5e-4fec-ab2b-968a361ffb56 tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2e60cf2931f04f0e8e911adea32979dd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7cb3e3496c1a45c8942c3678c0c299e5', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:25:05.069510 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-15d4fea5-be5e-4fec-ab2b-968a361ffb56 tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] 10.222.0.52 "GET /compute/v2.1/servers/e49199b7-e998-44fb-9995-e6a13fd44dc6" status: 200 len: 1265 microversion: 2.1 time: 0.111681 Aug 27 02:25:05.070084 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 162/326] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:25:04 2022] GET /compute/v2.1/servers/e49199b7-e998-44fb-9995-e6a13fd44dc6 => generated 1265 bytes in 114 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:25:05.211376 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-47c9492b-286e-40ad-bfd4-82995f834752 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:25:05.211947 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-47c9492b-286e-40ad-bfd4-82995f834752 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] [instance: 56dbfa6a-5b6f-4b82-977b-af735e6a6ac9] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:25:05.214277 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG dbcounter [-] [120863] Writing DB stats nova_cell0:SELECT=2 {{(pid=120863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 27 02:25:05.220066 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-47c9492b-286e-40ad-bfd4-82995f834752 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:25:05.220454 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-47c9492b-286e-40ad-bfd4-82995f834752 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:25:05.220914 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-47c9492b-286e-40ad-bfd4-82995f834752 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:25:05.221630 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-47c9492b-286e-40ad-bfd4-82995f834752 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:25:05.222080 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-47c9492b-286e-40ad-bfd4-82995f834752 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:25:05.222481 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-47c9492b-286e-40ad-bfd4-82995f834752 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:25:05.599434 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-47c9492b-286e-40ad-bfd4-82995f834752 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:25:05.607906 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-47c9492b-286e-40ad-bfd4-82995f834752 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:25:05.609315 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-47c9492b-286e-40ad-bfd4-82995f834752 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:25:05.610401 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-47c9492b-286e-40ad-bfd4-82995f834752 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] 10.222.0.52 "GET /compute/v2.1/servers/56dbfa6a-5b6f-4b82-977b-af735e6a6ac9" status: 200 len: 1407 microversion: 2.20 time: 0.403036 Aug 27 02:25:05.610959 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 165/327] 10.222.0.52 () {60 vars in 1277 bytes} [Sat Aug 27 02:25:05 2022] GET /compute/v2.1/servers/56dbfa6a-5b6f-4b82-977b-af735e6a6ac9 => generated 1407 bytes in 405 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 27 02:25:05.770468 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-f640eb72-7256-4d2f-94fd-72dbd72faf18 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Calling method '>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:25:05.771072 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-f640eb72-7256-4d2f-94fd-72dbd72faf18 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] [instance: e2bf7024-0779-4e08-844a-43b78187e770] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:25:05.777624 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-f640eb72-7256-4d2f-94fd-72dbd72faf18 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:25:05.778107 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-f640eb72-7256-4d2f-94fd-72dbd72faf18 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:25:05.778531 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-f640eb72-7256-4d2f-94fd-72dbd72faf18 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:25:05.779222 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-f640eb72-7256-4d2f-94fd-72dbd72faf18 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:25:05.779701 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-f640eb72-7256-4d2f-94fd-72dbd72faf18 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:25:05.780059 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-f640eb72-7256-4d2f-94fd-72dbd72faf18 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:25:05.911171 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-f640eb72-7256-4d2f-94fd-72dbd72faf18 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd945d512ae034ec08838d0a3afde2429', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '880ac4003248441c809a76cfd1b221eb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:25:05.919530 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-f640eb72-7256-4d2f-94fd-72dbd72faf18 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] 10.222.0.52 "GET /compute/v2.1/servers/e2bf7024-0779-4e08-844a-43b78187e770" status: 200 len: 1522 microversion: 2.1 time: 0.153521 Aug 27 02:25:05.920095 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 163/328] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:25:05 2022] GET /compute/v2.1/servers/e2bf7024-0779-4e08-844a-43b78187e770 => generated 1522 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:25:06.091233 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-906814d6-3839-43da-a8e2-eabe26a8e72a tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:25:06.091838 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-906814d6-3839-43da-a8e2-eabe26a8e72a tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] [instance: e49199b7-e998-44fb-9995-e6a13fd44dc6] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:25:06.098484 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-906814d6-3839-43da-a8e2-eabe26a8e72a tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:25:06.098968 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-906814d6-3839-43da-a8e2-eabe26a8e72a tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:25:06.099402 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-906814d6-3839-43da-a8e2-eabe26a8e72a tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:25:06.100096 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-906814d6-3839-43da-a8e2-eabe26a8e72a tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:25:06.100548 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-906814d6-3839-43da-a8e2-eabe26a8e72a tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:25:06.100979 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-906814d6-3839-43da-a8e2-eabe26a8e72a tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:25:06.168414 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-906814d6-3839-43da-a8e2-eabe26a8e72a tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2e60cf2931f04f0e8e911adea32979dd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7cb3e3496c1a45c8942c3678c0c299e5', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:25:06.176676 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-906814d6-3839-43da-a8e2-eabe26a8e72a tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] 10.222.0.52 "GET /compute/v2.1/servers/e49199b7-e998-44fb-9995-e6a13fd44dc6" status: 200 len: 1265 microversion: 2.1 time: 0.089342 Aug 27 02:25:06.177274 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 166/329] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:25:06 2022] GET /compute/v2.1/servers/e49199b7-e998-44fb-9995-e6a13fd44dc6 => generated 1265 bytes in 91 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:25:06.632692 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-60e53570-a966-42db-a4c5-79a8d00a49c7 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Calling method '>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:25:06.633329 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-60e53570-a966-42db-a4c5-79a8d00a49c7 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] [instance: 56dbfa6a-5b6f-4b82-977b-af735e6a6ac9] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:25:06.640146 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-60e53570-a966-42db-a4c5-79a8d00a49c7 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:25:06.640484 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-60e53570-a966-42db-a4c5-79a8d00a49c7 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:25:06.640882 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-60e53570-a966-42db-a4c5-79a8d00a49c7 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:25:06.641654 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-60e53570-a966-42db-a4c5-79a8d00a49c7 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:25:06.642066 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-60e53570-a966-42db-a4c5-79a8d00a49c7 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:25:06.642723 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-60e53570-a966-42db-a4c5-79a8d00a49c7 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:25:06.793831 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-60e53570-a966-42db-a4c5-79a8d00a49c7 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:25:06.802796 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-60e53570-a966-42db-a4c5-79a8d00a49c7 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:25:06.804228 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-60e53570-a966-42db-a4c5-79a8d00a49c7 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:25:06.805610 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-60e53570-a966-42db-a4c5-79a8d00a49c7 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] 10.222.0.52 "GET /compute/v2.1/servers/56dbfa6a-5b6f-4b82-977b-af735e6a6ac9" status: 200 len: 1407 microversion: 2.20 time: 0.176932 Aug 27 02:25:06.806175 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 164/330] 10.222.0.52 () {60 vars in 1277 bytes} [Sat Aug 27 02:25:06 2022] GET /compute/v2.1/servers/56dbfa6a-5b6f-4b82-977b-af735e6a6ac9 => generated 1407 bytes in 178 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 27 02:25:06.940698 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-97b6720d-60a5-41c3-a29e-129fa58d0911 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:25:06.941453 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-97b6720d-60a5-41c3-a29e-129fa58d0911 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] [instance: e2bf7024-0779-4e08-844a-43b78187e770] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:25:06.948582 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-97b6720d-60a5-41c3-a29e-129fa58d0911 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:25:06.949178 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-97b6720d-60a5-41c3-a29e-129fa58d0911 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:25:06.950003 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-97b6720d-60a5-41c3-a29e-129fa58d0911 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:25:06.950734 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-97b6720d-60a5-41c3-a29e-129fa58d0911 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:25:06.951219 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-97b6720d-60a5-41c3-a29e-129fa58d0911 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:25:06.951645 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-97b6720d-60a5-41c3-a29e-129fa58d0911 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:25:07.087403 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-97b6720d-60a5-41c3-a29e-129fa58d0911 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd945d512ae034ec08838d0a3afde2429', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '880ac4003248441c809a76cfd1b221eb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:25:07.090819 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG dbcounter [-] [120863] Writing DB stats nova_cell1:SELECT=100 {{(pid=120863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 27 02:25:07.096027 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-97b6720d-60a5-41c3-a29e-129fa58d0911 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] 10.222.0.52 "GET /compute/v2.1/servers/e2bf7024-0779-4e08-844a-43b78187e770" status: 200 len: 1522 microversion: 2.1 time: 0.159550 Aug 27 02:25:07.096596 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 167/331] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:25:06 2022] GET /compute/v2.1/servers/e2bf7024-0779-4e08-844a-43b78187e770 => generated 1522 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:25:07.197904 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-18e64747-b111-4fad-a4f3-75e8562a5c92 tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Calling method '>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:25:07.198504 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-18e64747-b111-4fad-a4f3-75e8562a5c92 tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] [instance: e49199b7-e998-44fb-9995-e6a13fd44dc6] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:25:07.205359 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-18e64747-b111-4fad-a4f3-75e8562a5c92 tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:25:07.205840 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-18e64747-b111-4fad-a4f3-75e8562a5c92 tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:25:07.206263 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-18e64747-b111-4fad-a4f3-75e8562a5c92 tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:25:07.207380 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-18e64747-b111-4fad-a4f3-75e8562a5c92 tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:25:07.207504 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-18e64747-b111-4fad-a4f3-75e8562a5c92 tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:25:07.207908 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-18e64747-b111-4fad-a4f3-75e8562a5c92 tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:25:07.349466 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-18e64747-b111-4fad-a4f3-75e8562a5c92 tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2e60cf2931f04f0e8e911adea32979dd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7cb3e3496c1a45c8942c3678c0c299e5', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:25:07.353297 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG dbcounter [-] [120864] Writing DB stats nova_cell1:SELECT=100 {{(pid=120864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 27 02:25:07.358775 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-18e64747-b111-4fad-a4f3-75e8562a5c92 tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] 10.222.0.52 "GET /compute/v2.1/servers/e49199b7-e998-44fb-9995-e6a13fd44dc6" status: 200 len: 1313 microversion: 2.1 time: 0.164752 Aug 27 02:25:07.359348 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 165/332] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:25:07 2022] GET /compute/v2.1/servers/e49199b7-e998-44fb-9995-e6a13fd44dc6 => generated 1313 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:25:07.830398 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-0d89fe48-f010-451b-a546-62ef0bc80121 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:25:07.831036 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-0d89fe48-f010-451b-a546-62ef0bc80121 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] [instance: 56dbfa6a-5b6f-4b82-977b-af735e6a6ac9] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:25:07.839406 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-0d89fe48-f010-451b-a546-62ef0bc80121 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:25:07.840192 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-0d89fe48-f010-451b-a546-62ef0bc80121 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:25:07.840921 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-0d89fe48-f010-451b-a546-62ef0bc80121 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:25:07.841959 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-0d89fe48-f010-451b-a546-62ef0bc80121 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:25:07.842570 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-0d89fe48-f010-451b-a546-62ef0bc80121 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:25:07.843147 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-0d89fe48-f010-451b-a546-62ef0bc80121 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:25:07.975756 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-0d89fe48-f010-451b-a546-62ef0bc80121 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:25:07.985450 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-0d89fe48-f010-451b-a546-62ef0bc80121 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:25:07.987104 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-0d89fe48-f010-451b-a546-62ef0bc80121 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:25:07.988461 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-0d89fe48-f010-451b-a546-62ef0bc80121 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] 10.222.0.52 "GET /compute/v2.1/servers/56dbfa6a-5b6f-4b82-977b-af735e6a6ac9" status: 200 len: 1407 microversion: 2.20 time: 0.162266 Aug 27 02:25:07.989164 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 168/333] 10.222.0.52 () {60 vars in 1277 bytes} [Sat Aug 27 02:25:07 2022] GET /compute/v2.1/servers/56dbfa6a-5b6f-4b82-977b-af735e6a6ac9 => generated 1407 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 27 02:25:08.112171 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [req-dd66c655-f156-4bed-a905-aa9d5f012522 req-ef6343ad-e99c-4822-8249-7bb305f6d034 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "e49199b7-e998-44fb-9995-e6a13fd44dc6", "tag": "55122ed4-7c38-4c0f-b4b9-a09312935cbc"}]} {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 27 02:25:08.117765 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-1d31e07f-f422-4eaf-bddc-35933329f77d tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:25:08.118624 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-1d31e07f-f422-4eaf-bddc-35933329f77d tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] [instance: e2bf7024-0779-4e08-844a-43b78187e770] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:25:08.124362 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [req-dd66c655-f156-4bed-a905-aa9d5f012522 req-ef6343ad-e99c-4822-8249-7bb305f6d034 service nova] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:25:08.124540 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-1d31e07f-f422-4eaf-bddc-35933329f77d tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:25:08.124882 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [req-dd66c655-f156-4bed-a905-aa9d5f012522 req-ef6343ad-e99c-4822-8249-7bb305f6d034 service nova] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:25:08.125038 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-1d31e07f-f422-4eaf-bddc-35933329f77d tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:25:08.125341 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [req-dd66c655-f156-4bed-a905-aa9d5f012522 req-ef6343ad-e99c-4822-8249-7bb305f6d034 service nova] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:25:08.125683 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-1d31e07f-f422-4eaf-bddc-35933329f77d tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:25:08.126732 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-1d31e07f-f422-4eaf-bddc-35933329f77d tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:25:08.127285 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-1d31e07f-f422-4eaf-bddc-35933329f77d tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:25:08.127769 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-1d31e07f-f422-4eaf-bddc-35933329f77d tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:25:08.143663 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.compute.server_external_events [req-dd66c655-f156-4bed-a905-aa9d5f012522 req-ef6343ad-e99c-4822-8249-7bb305f6d034 service nova] Creating event network-changed:55122ed4-7c38-4c0f-b4b9-a09312935cbc for instance e49199b7-e998-44fb-9995-e6a13fd44dc6 on hv-cinder-84630 Aug 27 02:25:08.150969 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [req-dd66c655-f156-4bed-a905-aa9d5f012522 req-ef6343ad-e99c-4822-8249-7bb305f6d034 service nova] 10.222.0.52 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.042191 Aug 27 02:25:08.152065 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 166/334] 10.222.0.52 () {64 vars in 1317 bytes} [Sat Aug 27 02:25:08 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 27 02:25:08.284022 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-1d31e07f-f422-4eaf-bddc-35933329f77d tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd945d512ae034ec08838d0a3afde2429', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '880ac4003248441c809a76cfd1b221eb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:25:08.292412 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-1d31e07f-f422-4eaf-bddc-35933329f77d tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] 10.222.0.52 "GET /compute/v2.1/servers/e2bf7024-0779-4e08-844a-43b78187e770" status: 200 len: 1522 microversion: 2.1 time: 0.178841 Aug 27 02:25:08.297640 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 169/335] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:25:08 2022] GET /compute/v2.1/servers/e2bf7024-0779-4e08-844a-43b78187e770 => generated 1522 bytes in 181 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:25:08.380539 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-dbc60f09-18cf-44ad-82b5-d46d615a5796 tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Calling method '>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:25:08.381447 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-dbc60f09-18cf-44ad-82b5-d46d615a5796 tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] [instance: e49199b7-e998-44fb-9995-e6a13fd44dc6] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:25:08.387694 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-dbc60f09-18cf-44ad-82b5-d46d615a5796 tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:25:08.388279 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-dbc60f09-18cf-44ad-82b5-d46d615a5796 tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:25:08.388836 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-dbc60f09-18cf-44ad-82b5-d46d615a5796 tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:25:08.389626 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-dbc60f09-18cf-44ad-82b5-d46d615a5796 tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:25:08.390315 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-dbc60f09-18cf-44ad-82b5-d46d615a5796 tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:25:08.390964 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-dbc60f09-18cf-44ad-82b5-d46d615a5796 tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:25:08.576588 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-dbc60f09-18cf-44ad-82b5-d46d615a5796 tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2e60cf2931f04f0e8e911adea32979dd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7cb3e3496c1a45c8942c3678c0c299e5', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:25:08.590085 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-dbc60f09-18cf-44ad-82b5-d46d615a5796 tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] 10.222.0.52 "GET /compute/v2.1/servers/e49199b7-e998-44fb-9995-e6a13fd44dc6" status: 200 len: 1313 microversion: 2.1 time: 0.214212 Aug 27 02:25:08.591222 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 167/336] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:25:08 2022] GET /compute/v2.1/servers/e49199b7-e998-44fb-9995-e6a13fd44dc6 => generated 1313 bytes in 216 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:25:09.019275 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-9b097ddd-9ae6-42fa-a327-3df3b0efc9f4 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:25:09.020365 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-9b097ddd-9ae6-42fa-a327-3df3b0efc9f4 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] [instance: 56dbfa6a-5b6f-4b82-977b-af735e6a6ac9] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:25:09.031477 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-9b097ddd-9ae6-42fa-a327-3df3b0efc9f4 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:25:09.032350 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-9b097ddd-9ae6-42fa-a327-3df3b0efc9f4 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:25:09.033204 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-9b097ddd-9ae6-42fa-a327-3df3b0efc9f4 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:25:09.034404 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-9b097ddd-9ae6-42fa-a327-3df3b0efc9f4 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:25:09.035195 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-9b097ddd-9ae6-42fa-a327-3df3b0efc9f4 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:25:09.035940 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-9b097ddd-9ae6-42fa-a327-3df3b0efc9f4 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:25:09.189740 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-9b097ddd-9ae6-42fa-a327-3df3b0efc9f4 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:25:09.198543 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-9b097ddd-9ae6-42fa-a327-3df3b0efc9f4 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:25:09.200310 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-9b097ddd-9ae6-42fa-a327-3df3b0efc9f4 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:25:09.201814 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-9b097ddd-9ae6-42fa-a327-3df3b0efc9f4 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] 10.222.0.52 "GET /compute/v2.1/servers/56dbfa6a-5b6f-4b82-977b-af735e6a6ac9" status: 200 len: 1407 microversion: 2.20 time: 0.193803 Aug 27 02:25:09.202362 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 170/337] 10.222.0.52 () {60 vars in 1277 bytes} [Sat Aug 27 02:25:09 2022] GET /compute/v2.1/servers/56dbfa6a-5b6f-4b82-977b-af735e6a6ac9 => generated 1407 bytes in 195 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 27 02:25:09.318664 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-89e1f2ab-758a-4196-832e-6bf5fc568cf4 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Calling method '>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:25:09.319399 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-89e1f2ab-758a-4196-832e-6bf5fc568cf4 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] [instance: e2bf7024-0779-4e08-844a-43b78187e770] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:25:09.326886 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-89e1f2ab-758a-4196-832e-6bf5fc568cf4 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:25:09.327662 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-89e1f2ab-758a-4196-832e-6bf5fc568cf4 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:25:09.328344 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-89e1f2ab-758a-4196-832e-6bf5fc568cf4 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:25:09.329414 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-89e1f2ab-758a-4196-832e-6bf5fc568cf4 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:25:09.330150 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-89e1f2ab-758a-4196-832e-6bf5fc568cf4 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:25:09.330887 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-89e1f2ab-758a-4196-832e-6bf5fc568cf4 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:25:09.469813 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-89e1f2ab-758a-4196-832e-6bf5fc568cf4 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd945d512ae034ec08838d0a3afde2429', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '880ac4003248441c809a76cfd1b221eb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:25:09.479151 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-89e1f2ab-758a-4196-832e-6bf5fc568cf4 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] 10.222.0.52 "GET /compute/v2.1/servers/e2bf7024-0779-4e08-844a-43b78187e770" status: 200 len: 1522 microversion: 2.1 time: 0.166696 Aug 27 02:25:09.479960 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 168/338] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:25:09 2022] GET /compute/v2.1/servers/e2bf7024-0779-4e08-844a-43b78187e770 => generated 1522 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:25:09.611806 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-db7f90ab-0e34-40b2-9daa-5f9586ac7606 tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:25:09.612429 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-db7f90ab-0e34-40b2-9daa-5f9586ac7606 tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] [instance: e49199b7-e998-44fb-9995-e6a13fd44dc6] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:25:09.618808 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-db7f90ab-0e34-40b2-9daa-5f9586ac7606 tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:25:09.619395 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-db7f90ab-0e34-40b2-9daa-5f9586ac7606 tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:25:09.619728 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-db7f90ab-0e34-40b2-9daa-5f9586ac7606 tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:25:09.620431 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-db7f90ab-0e34-40b2-9daa-5f9586ac7606 tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:25:09.620874 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-db7f90ab-0e34-40b2-9daa-5f9586ac7606 tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:25:09.621321 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-db7f90ab-0e34-40b2-9daa-5f9586ac7606 tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:25:09.764632 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-db7f90ab-0e34-40b2-9daa-5f9586ac7606 tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2e60cf2931f04f0e8e911adea32979dd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7cb3e3496c1a45c8942c3678c0c299e5', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:25:09.776457 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-db7f90ab-0e34-40b2-9daa-5f9586ac7606 tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] 10.222.0.52 "GET /compute/v2.1/servers/e49199b7-e998-44fb-9995-e6a13fd44dc6" status: 200 len: 1313 microversion: 2.1 time: 0.168503 Aug 27 02:25:09.777036 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 171/339] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:25:09 2022] GET /compute/v2.1/servers/e49199b7-e998-44fb-9995-e6a13fd44dc6 => generated 1313 bytes in 170 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:25:10.228578 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-08622c6d-40f5-43b6-87b7-9163912f62f5 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Calling method '>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:25:10.229480 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-08622c6d-40f5-43b6-87b7-9163912f62f5 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] [instance: 56dbfa6a-5b6f-4b82-977b-af735e6a6ac9] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:25:10.238937 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-08622c6d-40f5-43b6-87b7-9163912f62f5 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:25:10.239417 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-08622c6d-40f5-43b6-87b7-9163912f62f5 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:25:10.239848 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-08622c6d-40f5-43b6-87b7-9163912f62f5 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:25:10.240534 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-08622c6d-40f5-43b6-87b7-9163912f62f5 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:25:10.240982 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-08622c6d-40f5-43b6-87b7-9163912f62f5 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:25:10.241432 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-08622c6d-40f5-43b6-87b7-9163912f62f5 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:25:10.387174 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-08622c6d-40f5-43b6-87b7-9163912f62f5 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:25:10.397045 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-08622c6d-40f5-43b6-87b7-9163912f62f5 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:25:10.398528 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-08622c6d-40f5-43b6-87b7-9163912f62f5 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:25:10.399638 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-08622c6d-40f5-43b6-87b7-9163912f62f5 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] 10.222.0.52 "GET /compute/v2.1/servers/56dbfa6a-5b6f-4b82-977b-af735e6a6ac9" status: 200 len: 1407 microversion: 2.20 time: 0.178190 Aug 27 02:25:10.400179 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 169/340] 10.222.0.52 () {60 vars in 1277 bytes} [Sat Aug 27 02:25:10 2022] GET /compute/v2.1/servers/56dbfa6a-5b6f-4b82-977b-af735e6a6ac9 => generated 1407 bytes in 180 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 27 02:25:10.503526 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-fbafd45b-c392-455b-8a74-ca8380b85243 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:25:10.504276 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-fbafd45b-c392-455b-8a74-ca8380b85243 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] [instance: e2bf7024-0779-4e08-844a-43b78187e770] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:25:10.511694 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-fbafd45b-c392-455b-8a74-ca8380b85243 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:25:10.512357 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-fbafd45b-c392-455b-8a74-ca8380b85243 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:25:10.512807 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-fbafd45b-c392-455b-8a74-ca8380b85243 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:25:10.513741 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-fbafd45b-c392-455b-8a74-ca8380b85243 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:25:10.514204 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-fbafd45b-c392-455b-8a74-ca8380b85243 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:25:10.514631 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-fbafd45b-c392-455b-8a74-ca8380b85243 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:25:10.665799 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-fbafd45b-c392-455b-8a74-ca8380b85243 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd945d512ae034ec08838d0a3afde2429', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '880ac4003248441c809a76cfd1b221eb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:25:10.674098 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-fbafd45b-c392-455b-8a74-ca8380b85243 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] 10.222.0.52 "GET /compute/v2.1/servers/e2bf7024-0779-4e08-844a-43b78187e770" status: 200 len: 1522 microversion: 2.1 time: 0.174562 Aug 27 02:25:10.674759 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 172/341] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:25:10 2022] GET /compute/v2.1/servers/e2bf7024-0779-4e08-844a-43b78187e770 => generated 1522 bytes in 176 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:25:10.806289 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-ba00f524-b0cc-4872-9a36-9dec1cfa9385 tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Calling method '>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:25:10.806783 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-ba00f524-b0cc-4872-9a36-9dec1cfa9385 tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] [instance: e49199b7-e998-44fb-9995-e6a13fd44dc6] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:25:10.813261 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-ba00f524-b0cc-4872-9a36-9dec1cfa9385 tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:25:10.813726 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-ba00f524-b0cc-4872-9a36-9dec1cfa9385 tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:25:10.814143 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-ba00f524-b0cc-4872-9a36-9dec1cfa9385 tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:25:10.814824 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-ba00f524-b0cc-4872-9a36-9dec1cfa9385 tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:25:10.815268 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-ba00f524-b0cc-4872-9a36-9dec1cfa9385 tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:25:10.815677 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-ba00f524-b0cc-4872-9a36-9dec1cfa9385 tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:25:10.967510 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-ba00f524-b0cc-4872-9a36-9dec1cfa9385 tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2e60cf2931f04f0e8e911adea32979dd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7cb3e3496c1a45c8942c3678c0c299e5', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:25:10.976117 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-ba00f524-b0cc-4872-9a36-9dec1cfa9385 tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] 10.222.0.52 "GET /compute/v2.1/servers/e49199b7-e998-44fb-9995-e6a13fd44dc6" status: 200 len: 1313 microversion: 2.1 time: 0.173703 Aug 27 02:25:10.976693 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 170/342] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:25:10 2022] GET /compute/v2.1/servers/e49199b7-e998-44fb-9995-e6a13fd44dc6 => generated 1313 bytes in 178 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:25:11.422546 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-18df2073-fea8-4f2b-8639-48d769ed9d68 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:25:11.423138 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-18df2073-fea8-4f2b-8639-48d769ed9d68 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] [instance: 56dbfa6a-5b6f-4b82-977b-af735e6a6ac9] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:25:11.429882 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-18df2073-fea8-4f2b-8639-48d769ed9d68 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:25:11.430363 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-18df2073-fea8-4f2b-8639-48d769ed9d68 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:25:11.430841 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-18df2073-fea8-4f2b-8639-48d769ed9d68 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:25:11.431493 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-18df2073-fea8-4f2b-8639-48d769ed9d68 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:25:11.431938 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-18df2073-fea8-4f2b-8639-48d769ed9d68 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:25:11.432355 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-18df2073-fea8-4f2b-8639-48d769ed9d68 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:25:11.581748 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-18df2073-fea8-4f2b-8639-48d769ed9d68 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:25:11.590370 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-18df2073-fea8-4f2b-8639-48d769ed9d68 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:25:11.591808 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-18df2073-fea8-4f2b-8639-48d769ed9d68 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:25:11.592916 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-18df2073-fea8-4f2b-8639-48d769ed9d68 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] 10.222.0.52 "GET /compute/v2.1/servers/56dbfa6a-5b6f-4b82-977b-af735e6a6ac9" status: 200 len: 1407 microversion: 2.20 time: 0.174325 Aug 27 02:25:11.593501 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 173/343] 10.222.0.52 () {60 vars in 1277 bytes} [Sat Aug 27 02:25:11 2022] GET /compute/v2.1/servers/56dbfa6a-5b6f-4b82-977b-af735e6a6ac9 => generated 1407 bytes in 176 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 27 02:25:11.697051 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-1bf47cc0-903e-4698-896d-39debe393027 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Calling method '>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:25:11.697633 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-1bf47cc0-903e-4698-896d-39debe393027 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] [instance: e2bf7024-0779-4e08-844a-43b78187e770] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:25:11.704310 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-1bf47cc0-903e-4698-896d-39debe393027 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:25:11.705060 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-1bf47cc0-903e-4698-896d-39debe393027 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:25:11.705258 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-1bf47cc0-903e-4698-896d-39debe393027 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:25:11.705959 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-1bf47cc0-903e-4698-896d-39debe393027 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:25:11.706423 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-1bf47cc0-903e-4698-896d-39debe393027 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:25:11.706843 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-1bf47cc0-903e-4698-896d-39debe393027 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:25:11.843502 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-1bf47cc0-903e-4698-896d-39debe393027 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd945d512ae034ec08838d0a3afde2429', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '880ac4003248441c809a76cfd1b221eb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:25:11.853662 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-1bf47cc0-903e-4698-896d-39debe393027 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] 10.222.0.52 "GET /compute/v2.1/servers/e2bf7024-0779-4e08-844a-43b78187e770" status: 200 len: 1522 microversion: 2.1 time: 0.160937 Aug 27 02:25:11.854234 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 171/344] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:25:11 2022] GET /compute/v2.1/servers/e2bf7024-0779-4e08-844a-43b78187e770 => generated 1522 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:25:11.997558 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-e5bfbda0-4c2f-48da-9fc7-67bcf1466ad6 tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:25:11.998194 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-e5bfbda0-4c2f-48da-9fc7-67bcf1466ad6 tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] [instance: e49199b7-e998-44fb-9995-e6a13fd44dc6] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:25:12.004426 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-e5bfbda0-4c2f-48da-9fc7-67bcf1466ad6 tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:25:12.004905 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-e5bfbda0-4c2f-48da-9fc7-67bcf1466ad6 tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:25:12.005366 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-e5bfbda0-4c2f-48da-9fc7-67bcf1466ad6 tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:25:12.006060 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-e5bfbda0-4c2f-48da-9fc7-67bcf1466ad6 tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:25:12.006512 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-e5bfbda0-4c2f-48da-9fc7-67bcf1466ad6 tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:25:12.006928 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-e5bfbda0-4c2f-48da-9fc7-67bcf1466ad6 tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:25:12.148641 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-e5bfbda0-4c2f-48da-9fc7-67bcf1466ad6 tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2e60cf2931f04f0e8e911adea32979dd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7cb3e3496c1a45c8942c3678c0c299e5', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:25:12.156719 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-e5bfbda0-4c2f-48da-9fc7-67bcf1466ad6 tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] 10.222.0.52 "GET /compute/v2.1/servers/e49199b7-e998-44fb-9995-e6a13fd44dc6" status: 200 len: 1313 microversion: 2.1 time: 0.163137 Aug 27 02:25:12.157327 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 174/345] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:25:11 2022] GET /compute/v2.1/servers/e49199b7-e998-44fb-9995-e6a13fd44dc6 => generated 1313 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:25:12.618272 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-91b7ecca-180f-496b-92bb-612fde4a67db tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Calling method '>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:25:12.618874 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-91b7ecca-180f-496b-92bb-612fde4a67db tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] [instance: 56dbfa6a-5b6f-4b82-977b-af735e6a6ac9] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:25:12.625936 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-91b7ecca-180f-496b-92bb-612fde4a67db tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:25:12.628875 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-91b7ecca-180f-496b-92bb-612fde4a67db tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:25:12.628875 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-91b7ecca-180f-496b-92bb-612fde4a67db tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:25:12.628875 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-91b7ecca-180f-496b-92bb-612fde4a67db tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:25:12.628875 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-91b7ecca-180f-496b-92bb-612fde4a67db tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:25:12.629429 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-91b7ecca-180f-496b-92bb-612fde4a67db tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:25:12.765528 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-91b7ecca-180f-496b-92bb-612fde4a67db tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:25:12.775164 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-91b7ecca-180f-496b-92bb-612fde4a67db tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:25:12.776716 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-91b7ecca-180f-496b-92bb-612fde4a67db tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:25:12.777848 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-91b7ecca-180f-496b-92bb-612fde4a67db tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] 10.222.0.52 "GET /compute/v2.1/servers/56dbfa6a-5b6f-4b82-977b-af735e6a6ac9" status: 200 len: 1407 microversion: 2.20 time: 0.163560 Aug 27 02:25:12.778413 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 172/346] 10.222.0.52 () {60 vars in 1277 bytes} [Sat Aug 27 02:25:12 2022] GET /compute/v2.1/servers/56dbfa6a-5b6f-4b82-977b-af735e6a6ac9 => generated 1407 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 27 02:25:12.874994 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-355e45ef-4461-4841-87b1-295ab11af84e tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:25:12.875561 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-355e45ef-4461-4841-87b1-295ab11af84e tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] [instance: e2bf7024-0779-4e08-844a-43b78187e770] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:25:12.882718 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-355e45ef-4461-4841-87b1-295ab11af84e tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:25:12.883239 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-355e45ef-4461-4841-87b1-295ab11af84e tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:25:12.883678 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-355e45ef-4461-4841-87b1-295ab11af84e tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:25:12.884379 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-355e45ef-4461-4841-87b1-295ab11af84e tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:25:12.884836 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-355e45ef-4461-4841-87b1-295ab11af84e tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:25:12.885291 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-355e45ef-4461-4841-87b1-295ab11af84e tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:25:13.048447 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-355e45ef-4461-4841-87b1-295ab11af84e tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd945d512ae034ec08838d0a3afde2429', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '880ac4003248441c809a76cfd1b221eb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:25:13.057203 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-355e45ef-4461-4841-87b1-295ab11af84e tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] 10.222.0.52 "GET /compute/v2.1/servers/e2bf7024-0779-4e08-844a-43b78187e770" status: 200 len: 1522 microversion: 2.1 time: 0.186326 Aug 27 02:25:13.057760 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 175/347] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:25:12 2022] GET /compute/v2.1/servers/e2bf7024-0779-4e08-844a-43b78187e770 => generated 1522 bytes in 188 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:25:13.180613 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-84a516dd-7358-4b23-8936-d77b874a14e9 tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Calling method '>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:25:13.181150 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-84a516dd-7358-4b23-8936-d77b874a14e9 tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] [instance: e49199b7-e998-44fb-9995-e6a13fd44dc6] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:25:13.190362 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-84a516dd-7358-4b23-8936-d77b874a14e9 tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:25:13.190985 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-84a516dd-7358-4b23-8936-d77b874a14e9 tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:25:13.191499 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-84a516dd-7358-4b23-8936-d77b874a14e9 tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:25:13.192285 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-84a516dd-7358-4b23-8936-d77b874a14e9 tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:25:13.192703 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-84a516dd-7358-4b23-8936-d77b874a14e9 tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:25:13.193237 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-84a516dd-7358-4b23-8936-d77b874a14e9 tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:25:13.327955 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-84a516dd-7358-4b23-8936-d77b874a14e9 tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2e60cf2931f04f0e8e911adea32979dd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7cb3e3496c1a45c8942c3678c0c299e5', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:25:13.337319 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-84a516dd-7358-4b23-8936-d77b874a14e9 tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] 10.222.0.52 "GET /compute/v2.1/servers/e49199b7-e998-44fb-9995-e6a13fd44dc6" status: 200 len: 1313 microversion: 2.1 time: 0.162019 Aug 27 02:25:13.337868 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 173/348] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:25:13 2022] GET /compute/v2.1/servers/e49199b7-e998-44fb-9995-e6a13fd44dc6 => generated 1313 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:25:13.800615 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-a2c3bcc3-b6f1-498c-b386-8f0d95d0e459 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:25:13.801241 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-a2c3bcc3-b6f1-498c-b386-8f0d95d0e459 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] [instance: 56dbfa6a-5b6f-4b82-977b-af735e6a6ac9] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:25:13.807968 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-a2c3bcc3-b6f1-498c-b386-8f0d95d0e459 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:25:13.808449 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-a2c3bcc3-b6f1-498c-b386-8f0d95d0e459 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:25:13.808882 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-a2c3bcc3-b6f1-498c-b386-8f0d95d0e459 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:25:13.809787 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-a2c3bcc3-b6f1-498c-b386-8f0d95d0e459 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:25:13.810235 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-a2c3bcc3-b6f1-498c-b386-8f0d95d0e459 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:25:13.810655 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-a2c3bcc3-b6f1-498c-b386-8f0d95d0e459 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:25:13.948519 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-a2c3bcc3-b6f1-498c-b386-8f0d95d0e459 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:25:13.960742 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-a2c3bcc3-b6f1-498c-b386-8f0d95d0e459 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:25:13.962772 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-a2c3bcc3-b6f1-498c-b386-8f0d95d0e459 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:25:13.964399 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-a2c3bcc3-b6f1-498c-b386-8f0d95d0e459 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] 10.222.0.52 "GET /compute/v2.1/servers/56dbfa6a-5b6f-4b82-977b-af735e6a6ac9" status: 200 len: 1407 microversion: 2.20 time: 0.167536 Aug 27 02:25:13.965370 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 176/349] 10.222.0.52 () {60 vars in 1277 bytes} [Sat Aug 27 02:25:13 2022] GET /compute/v2.1/servers/56dbfa6a-5b6f-4b82-977b-af735e6a6ac9 => generated 1407 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 27 02:25:14.079532 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-b0fc39e3-c4f1-453e-aa92-09e8f5d484ea tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Calling method '>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:25:14.080154 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-b0fc39e3-c4f1-453e-aa92-09e8f5d484ea tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] [instance: e2bf7024-0779-4e08-844a-43b78187e770] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:25:14.086476 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-b0fc39e3-c4f1-453e-aa92-09e8f5d484ea tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:25:14.087241 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-b0fc39e3-c4f1-453e-aa92-09e8f5d484ea tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:25:14.087670 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-b0fc39e3-c4f1-453e-aa92-09e8f5d484ea tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:25:14.088342 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-b0fc39e3-c4f1-453e-aa92-09e8f5d484ea tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:25:14.088820 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-b0fc39e3-c4f1-453e-aa92-09e8f5d484ea tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:25:14.089289 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-b0fc39e3-c4f1-453e-aa92-09e8f5d484ea tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:25:14.233571 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-b0fc39e3-c4f1-453e-aa92-09e8f5d484ea tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd945d512ae034ec08838d0a3afde2429', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '880ac4003248441c809a76cfd1b221eb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:25:14.241588 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-b0fc39e3-c4f1-453e-aa92-09e8f5d484ea tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] 10.222.0.52 "GET /compute/v2.1/servers/e2bf7024-0779-4e08-844a-43b78187e770" status: 200 len: 1522 microversion: 2.1 time: 0.166775 Aug 27 02:25:14.242149 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 174/350] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:25:14 2022] GET /compute/v2.1/servers/e2bf7024-0779-4e08-844a-43b78187e770 => generated 1522 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:25:14.363006 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-4f1523ca-37d5-4e3f-90c4-e4ee2a7a9402 tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:25:14.363731 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-4f1523ca-37d5-4e3f-90c4-e4ee2a7a9402 tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] [instance: e49199b7-e998-44fb-9995-e6a13fd44dc6] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:25:14.370203 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-4f1523ca-37d5-4e3f-90c4-e4ee2a7a9402 tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:25:14.370689 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-4f1523ca-37d5-4e3f-90c4-e4ee2a7a9402 tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:25:14.371126 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-4f1523ca-37d5-4e3f-90c4-e4ee2a7a9402 tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:25:14.371849 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-4f1523ca-37d5-4e3f-90c4-e4ee2a7a9402 tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:25:14.372304 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-4f1523ca-37d5-4e3f-90c4-e4ee2a7a9402 tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:25:14.372728 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-4f1523ca-37d5-4e3f-90c4-e4ee2a7a9402 tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:25:14.520051 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-4f1523ca-37d5-4e3f-90c4-e4ee2a7a9402 tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2e60cf2931f04f0e8e911adea32979dd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7cb3e3496c1a45c8942c3678c0c299e5', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:25:14.527846 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-4f1523ca-37d5-4e3f-90c4-e4ee2a7a9402 tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] 10.222.0.52 "GET /compute/v2.1/servers/e49199b7-e998-44fb-9995-e6a13fd44dc6" status: 200 len: 1313 microversion: 2.1 time: 0.168769 Aug 27 02:25:14.528438 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 177/351] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:25:14 2022] GET /compute/v2.1/servers/e49199b7-e998-44fb-9995-e6a13fd44dc6 => generated 1313 bytes in 170 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:25:14.991561 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-6f825fce-3c04-48a0-826f-1411e18a554e tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Calling method '>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:25:14.992338 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-6f825fce-3c04-48a0-826f-1411e18a554e tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] [instance: 56dbfa6a-5b6f-4b82-977b-af735e6a6ac9] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:25:15.000786 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-6f825fce-3c04-48a0-826f-1411e18a554e tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:25:15.001470 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-6f825fce-3c04-48a0-826f-1411e18a554e tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:25:15.002036 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-6f825fce-3c04-48a0-826f-1411e18a554e tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:25:15.002926 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-6f825fce-3c04-48a0-826f-1411e18a554e tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:25:15.003500 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-6f825fce-3c04-48a0-826f-1411e18a554e tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:25:15.004220 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-6f825fce-3c04-48a0-826f-1411e18a554e tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:25:15.145620 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-6f825fce-3c04-48a0-826f-1411e18a554e tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:25:15.154009 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-6f825fce-3c04-48a0-826f-1411e18a554e tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:25:15.157792 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-6f825fce-3c04-48a0-826f-1411e18a554e tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:25:15.157792 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-6f825fce-3c04-48a0-826f-1411e18a554e tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] 10.222.0.52 "GET /compute/v2.1/servers/56dbfa6a-5b6f-4b82-977b-af735e6a6ac9" status: 200 len: 1407 microversion: 2.20 time: 0.170397 Aug 27 02:25:15.167778 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 175/352] 10.222.0.52 () {60 vars in 1277 bytes} [Sat Aug 27 02:25:14 2022] GET /compute/v2.1/servers/56dbfa6a-5b6f-4b82-977b-af735e6a6ac9 => generated 1407 bytes in 182 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 27 02:25:15.263309 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-ee1cb959-7478-4727-a654-9329d6a179c3 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:25:15.264037 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-ee1cb959-7478-4727-a654-9329d6a179c3 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] [instance: e2bf7024-0779-4e08-844a-43b78187e770] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:25:15.272528 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-ee1cb959-7478-4727-a654-9329d6a179c3 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:25:15.273023 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-ee1cb959-7478-4727-a654-9329d6a179c3 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:25:15.273499 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-ee1cb959-7478-4727-a654-9329d6a179c3 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:25:15.274207 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-ee1cb959-7478-4727-a654-9329d6a179c3 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:25:15.274655 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-ee1cb959-7478-4727-a654-9329d6a179c3 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:25:15.275074 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-ee1cb959-7478-4727-a654-9329d6a179c3 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:25:15.412288 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-ee1cb959-7478-4727-a654-9329d6a179c3 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd945d512ae034ec08838d0a3afde2429', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '880ac4003248441c809a76cfd1b221eb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:25:15.420613 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-ee1cb959-7478-4727-a654-9329d6a179c3 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] 10.222.0.52 "GET /compute/v2.1/servers/e2bf7024-0779-4e08-844a-43b78187e770" status: 200 len: 1522 microversion: 2.1 time: 0.161100 Aug 27 02:25:15.421194 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 178/353] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:25:15 2022] GET /compute/v2.1/servers/e2bf7024-0779-4e08-844a-43b78187e770 => generated 1522 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:25:15.550130 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-8fa38202-b9d9-45a7-b859-35f9fcc65a17 tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Calling method '>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:25:15.550775 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-8fa38202-b9d9-45a7-b859-35f9fcc65a17 tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] [instance: e49199b7-e998-44fb-9995-e6a13fd44dc6] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:25:15.557431 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-8fa38202-b9d9-45a7-b859-35f9fcc65a17 tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:25:15.557913 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-8fa38202-b9d9-45a7-b859-35f9fcc65a17 tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:25:15.558345 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-8fa38202-b9d9-45a7-b859-35f9fcc65a17 tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:25:15.559032 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-8fa38202-b9d9-45a7-b859-35f9fcc65a17 tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:25:15.559481 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-8fa38202-b9d9-45a7-b859-35f9fcc65a17 tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:25:15.559897 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-8fa38202-b9d9-45a7-b859-35f9fcc65a17 tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:25:15.703194 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-8fa38202-b9d9-45a7-b859-35f9fcc65a17 tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2e60cf2931f04f0e8e911adea32979dd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7cb3e3496c1a45c8942c3678c0c299e5', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:25:15.716033 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-8fa38202-b9d9-45a7-b859-35f9fcc65a17 tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] 10.222.0.52 "GET /compute/v2.1/servers/e49199b7-e998-44fb-9995-e6a13fd44dc6" status: 200 len: 1313 microversion: 2.1 time: 0.169674 Aug 27 02:25:15.716928 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 176/354] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:25:15 2022] GET /compute/v2.1/servers/e49199b7-e998-44fb-9995-e6a13fd44dc6 => generated 1313 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:25:16.181009 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-3d9a9252-e8bd-483b-a2b2-1ef4ad78b1c3 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:25:16.181656 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-3d9a9252-e8bd-483b-a2b2-1ef4ad78b1c3 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] [instance: 56dbfa6a-5b6f-4b82-977b-af735e6a6ac9] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:25:16.187945 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-3d9a9252-e8bd-483b-a2b2-1ef4ad78b1c3 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:25:16.188343 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-3d9a9252-e8bd-483b-a2b2-1ef4ad78b1c3 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:25:16.188782 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-3d9a9252-e8bd-483b-a2b2-1ef4ad78b1c3 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:25:16.189490 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-3d9a9252-e8bd-483b-a2b2-1ef4ad78b1c3 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:25:16.189947 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-3d9a9252-e8bd-483b-a2b2-1ef4ad78b1c3 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:25:16.190367 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-3d9a9252-e8bd-483b-a2b2-1ef4ad78b1c3 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:25:16.326839 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-3d9a9252-e8bd-483b-a2b2-1ef4ad78b1c3 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:25:16.338555 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-3d9a9252-e8bd-483b-a2b2-1ef4ad78b1c3 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:25:16.340793 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-3d9a9252-e8bd-483b-a2b2-1ef4ad78b1c3 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:25:16.342490 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-3d9a9252-e8bd-483b-a2b2-1ef4ad78b1c3 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] 10.222.0.52 "GET /compute/v2.1/servers/56dbfa6a-5b6f-4b82-977b-af735e6a6ac9" status: 200 len: 1407 microversion: 2.20 time: 0.165256 Aug 27 02:25:16.343300 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 179/355] 10.222.0.52 () {60 vars in 1277 bytes} [Sat Aug 27 02:25:16 2022] GET /compute/v2.1/servers/56dbfa6a-5b6f-4b82-977b-af735e6a6ac9 => generated 1407 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 27 02:25:16.443613 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-b4a2651f-5ccc-4014-98ea-7d38e156952c tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Calling method '>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:25:16.444182 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-b4a2651f-5ccc-4014-98ea-7d38e156952c tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] [instance: e2bf7024-0779-4e08-844a-43b78187e770] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:25:16.451674 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-b4a2651f-5ccc-4014-98ea-7d38e156952c tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:25:16.452353 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-b4a2651f-5ccc-4014-98ea-7d38e156952c tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:25:16.452911 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-b4a2651f-5ccc-4014-98ea-7d38e156952c tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:25:16.453910 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-b4a2651f-5ccc-4014-98ea-7d38e156952c tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:25:16.454408 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-b4a2651f-5ccc-4014-98ea-7d38e156952c tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:25:16.454883 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-b4a2651f-5ccc-4014-98ea-7d38e156952c tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:25:16.596848 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-b4a2651f-5ccc-4014-98ea-7d38e156952c tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd945d512ae034ec08838d0a3afde2429', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '880ac4003248441c809a76cfd1b221eb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:25:16.611567 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-b4a2651f-5ccc-4014-98ea-7d38e156952c tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] 10.222.0.52 "GET /compute/v2.1/servers/e2bf7024-0779-4e08-844a-43b78187e770" status: 200 len: 1522 microversion: 2.1 time: 0.172753 Aug 27 02:25:16.612144 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 177/356] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:25:16 2022] GET /compute/v2.1/servers/e2bf7024-0779-4e08-844a-43b78187e770 => generated 1522 bytes in 174 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:25:16.744559 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-e31c6e1c-aacb-4ec4-8135-53ca76c71b2a tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:25:16.745226 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-e31c6e1c-aacb-4ec4-8135-53ca76c71b2a tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] [instance: e49199b7-e998-44fb-9995-e6a13fd44dc6] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:25:16.751252 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-e31c6e1c-aacb-4ec4-8135-53ca76c71b2a tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:25:16.751732 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-e31c6e1c-aacb-4ec4-8135-53ca76c71b2a tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:25:16.752161 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-e31c6e1c-aacb-4ec4-8135-53ca76c71b2a tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:25:16.752843 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-e31c6e1c-aacb-4ec4-8135-53ca76c71b2a tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:25:16.753401 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-e31c6e1c-aacb-4ec4-8135-53ca76c71b2a tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:25:16.753900 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-e31c6e1c-aacb-4ec4-8135-53ca76c71b2a tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:25:16.895838 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-e31c6e1c-aacb-4ec4-8135-53ca76c71b2a tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2e60cf2931f04f0e8e911adea32979dd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7cb3e3496c1a45c8942c3678c0c299e5', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:25:16.903403 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-e31c6e1c-aacb-4ec4-8135-53ca76c71b2a tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] 10.222.0.52 "GET /compute/v2.1/servers/e49199b7-e998-44fb-9995-e6a13fd44dc6" status: 200 len: 1313 microversion: 2.1 time: 0.162769 Aug 27 02:25:16.903959 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 180/357] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:25:16 2022] GET /compute/v2.1/servers/e49199b7-e998-44fb-9995-e6a13fd44dc6 => generated 1313 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:25:17.145321 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [req-4cdad1c2-823b-4d0a-be80-ba77620662c5 req-d08fc29b-b2d8-4b4b-889c-f2b990dac6f5 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "56dbfa6a-5b6f-4b82-977b-af735e6a6ac9", "name": "network-vif-plugged", "status": "completed", "tag": "520bd5fb-723a-4570-83ca-72759c588707"}]} {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 27 02:25:17.156334 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [req-4cdad1c2-823b-4d0a-be80-ba77620662c5 req-d08fc29b-b2d8-4b4b-889c-f2b990dac6f5 service nova] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:25:17.156810 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [req-4cdad1c2-823b-4d0a-be80-ba77620662c5 req-d08fc29b-b2d8-4b4b-889c-f2b990dac6f5 service nova] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:25:17.157361 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [req-4cdad1c2-823b-4d0a-be80-ba77620662c5 req-d08fc29b-b2d8-4b4b-889c-f2b990dac6f5 service nova] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:25:17.169700 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.compute.server_external_events [req-4cdad1c2-823b-4d0a-be80-ba77620662c5 req-d08fc29b-b2d8-4b4b-889c-f2b990dac6f5 service nova] Creating event network-vif-plugged:520bd5fb-723a-4570-83ca-72759c588707 for instance 56dbfa6a-5b6f-4b82-977b-af735e6a6ac9 on hv-cinder-84630 Aug 27 02:25:17.174896 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [req-4cdad1c2-823b-4d0a-be80-ba77620662c5 req-d08fc29b-b2d8-4b4b-889c-f2b990dac6f5 service nova] 10.222.0.52 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.033355 Aug 27 02:25:17.175648 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 178/358] 10.222.0.52 () {64 vars in 1317 bytes} [Sat Aug 27 02:25:17 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 35 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 27 02:25:17.365743 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-71ed6fbe-8015-41e9-95b9-9d0376b83f56 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:25:17.366278 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-71ed6fbe-8015-41e9-95b9-9d0376b83f56 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] [instance: 56dbfa6a-5b6f-4b82-977b-af735e6a6ac9] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:25:17.372937 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-71ed6fbe-8015-41e9-95b9-9d0376b83f56 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:25:17.373459 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-71ed6fbe-8015-41e9-95b9-9d0376b83f56 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:25:17.373893 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-71ed6fbe-8015-41e9-95b9-9d0376b83f56 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:25:17.374604 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-71ed6fbe-8015-41e9-95b9-9d0376b83f56 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:25:17.375063 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-71ed6fbe-8015-41e9-95b9-9d0376b83f56 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:25:17.375503 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-71ed6fbe-8015-41e9-95b9-9d0376b83f56 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:25:17.531426 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-71ed6fbe-8015-41e9-95b9-9d0376b83f56 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:25:17.539772 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-71ed6fbe-8015-41e9-95b9-9d0376b83f56 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:25:17.541264 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-71ed6fbe-8015-41e9-95b9-9d0376b83f56 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:25:17.542418 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-71ed6fbe-8015-41e9-95b9-9d0376b83f56 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] 10.222.0.52 "GET /compute/v2.1/servers/56dbfa6a-5b6f-4b82-977b-af735e6a6ac9" status: 200 len: 1407 microversion: 2.20 time: 0.181050 Aug 27 02:25:17.542948 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 181/359] 10.222.0.52 () {60 vars in 1277 bytes} [Sat Aug 27 02:25:17 2022] GET /compute/v2.1/servers/56dbfa6a-5b6f-4b82-977b-af735e6a6ac9 => generated 1407 bytes in 182 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 27 02:25:17.638714 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-9521b92f-36aa-43cc-9dca-8b9a5ed02039 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Calling method '>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:25:17.639302 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-9521b92f-36aa-43cc-9dca-8b9a5ed02039 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] [instance: e2bf7024-0779-4e08-844a-43b78187e770] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:25:17.647571 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-9521b92f-36aa-43cc-9dca-8b9a5ed02039 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:25:17.648407 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-9521b92f-36aa-43cc-9dca-8b9a5ed02039 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:25:17.649087 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-9521b92f-36aa-43cc-9dca-8b9a5ed02039 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:25:17.650000 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-9521b92f-36aa-43cc-9dca-8b9a5ed02039 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:25:17.650451 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-9521b92f-36aa-43cc-9dca-8b9a5ed02039 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:25:17.650869 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-9521b92f-36aa-43cc-9dca-8b9a5ed02039 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:25:17.795093 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-9521b92f-36aa-43cc-9dca-8b9a5ed02039 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd945d512ae034ec08838d0a3afde2429', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '880ac4003248441c809a76cfd1b221eb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:25:17.806546 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-9521b92f-36aa-43cc-9dca-8b9a5ed02039 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] 10.222.0.52 "GET /compute/v2.1/servers/e2bf7024-0779-4e08-844a-43b78187e770" status: 200 len: 1522 microversion: 2.1 time: 0.171795 Aug 27 02:25:17.807291 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 179/360] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:25:17 2022] GET /compute/v2.1/servers/e2bf7024-0779-4e08-844a-43b78187e770 => generated 1522 bytes in 174 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 27 02:25:17.923793 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-3cf3a37a-7a12-461d-8dc8-401fdbfcf868 tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:25:17.924548 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-3cf3a37a-7a12-461d-8dc8-401fdbfcf868 tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] [instance: e49199b7-e998-44fb-9995-e6a13fd44dc6] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:25:17.931202 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-3cf3a37a-7a12-461d-8dc8-401fdbfcf868 tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:25:17.931687 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-3cf3a37a-7a12-461d-8dc8-401fdbfcf868 tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:25:17.932116 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-3cf3a37a-7a12-461d-8dc8-401fdbfcf868 tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:25:17.932799 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-3cf3a37a-7a12-461d-8dc8-401fdbfcf868 tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:25:17.933279 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-3cf3a37a-7a12-461d-8dc8-401fdbfcf868 tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:25:17.933697 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-3cf3a37a-7a12-461d-8dc8-401fdbfcf868 tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:25:18.075594 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-3cf3a37a-7a12-461d-8dc8-401fdbfcf868 tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2e60cf2931f04f0e8e911adea32979dd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7cb3e3496c1a45c8942c3678c0c299e5', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:25:18.085465 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-3cf3a37a-7a12-461d-8dc8-401fdbfcf868 tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] 10.222.0.52 "GET /compute/v2.1/servers/e49199b7-e998-44fb-9995-e6a13fd44dc6" status: 200 len: 1313 microversion: 2.1 time: 0.165092 Aug 27 02:25:18.098925 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 182/361] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:25:17 2022] GET /compute/v2.1/servers/e49199b7-e998-44fb-9995-e6a13fd44dc6 => generated 1313 bytes in 179 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:25:18.567065 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-508c790b-b806-42e0-934c-2a8bef427e8d tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Calling method '>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:25:18.568220 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-508c790b-b806-42e0-934c-2a8bef427e8d tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] [instance: 56dbfa6a-5b6f-4b82-977b-af735e6a6ac9] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:25:18.574405 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-508c790b-b806-42e0-934c-2a8bef427e8d tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:25:18.575239 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-508c790b-b806-42e0-934c-2a8bef427e8d tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:25:18.575671 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-508c790b-b806-42e0-934c-2a8bef427e8d tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:25:18.576366 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-508c790b-b806-42e0-934c-2a8bef427e8d tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:25:18.576899 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-508c790b-b806-42e0-934c-2a8bef427e8d tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:25:18.577243 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-508c790b-b806-42e0-934c-2a8bef427e8d tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:25:18.714504 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-508c790b-b806-42e0-934c-2a8bef427e8d tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:25:18.722713 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-508c790b-b806-42e0-934c-2a8bef427e8d tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:25:18.724250 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-508c790b-b806-42e0-934c-2a8bef427e8d tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:25:18.725377 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-508c790b-b806-42e0-934c-2a8bef427e8d tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] 10.222.0.52 "GET /compute/v2.1/servers/56dbfa6a-5b6f-4b82-977b-af735e6a6ac9" status: 200 len: 1407 microversion: 2.20 time: 0.163628 Aug 27 02:25:18.725929 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 180/362] 10.222.0.52 () {60 vars in 1277 bytes} [Sat Aug 27 02:25:18 2022] GET /compute/v2.1/servers/56dbfa6a-5b6f-4b82-977b-af735e6a6ac9 => generated 1407 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 27 02:25:18.828414 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-2bf94823-dcfc-4b80-b2db-72784c98ff73 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:25:18.829054 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-2bf94823-dcfc-4b80-b2db-72784c98ff73 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] [instance: e2bf7024-0779-4e08-844a-43b78187e770] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:25:18.836565 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-2bf94823-dcfc-4b80-b2db-72784c98ff73 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:25:18.837055 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-2bf94823-dcfc-4b80-b2db-72784c98ff73 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:25:18.837534 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-2bf94823-dcfc-4b80-b2db-72784c98ff73 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:25:18.838241 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-2bf94823-dcfc-4b80-b2db-72784c98ff73 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:25:18.838896 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-2bf94823-dcfc-4b80-b2db-72784c98ff73 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:25:18.839315 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-2bf94823-dcfc-4b80-b2db-72784c98ff73 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:25:19.025925 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-2bf94823-dcfc-4b80-b2db-72784c98ff73 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd945d512ae034ec08838d0a3afde2429', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '880ac4003248441c809a76cfd1b221eb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:25:19.034792 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-2bf94823-dcfc-4b80-b2db-72784c98ff73 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] 10.222.0.52 "GET /compute/v2.1/servers/e2bf7024-0779-4e08-844a-43b78187e770" status: 200 len: 1522 microversion: 2.1 time: 0.210615 Aug 27 02:25:19.036281 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 183/363] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:25:18 2022] GET /compute/v2.1/servers/e2bf7024-0779-4e08-844a-43b78187e770 => generated 1522 bytes in 213 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:25:19.109017 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-c538ad5c-ccc8-443b-b4b3-11595d9df52e tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Calling method '>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:25:19.109947 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-c538ad5c-ccc8-443b-b4b3-11595d9df52e tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] [instance: e49199b7-e998-44fb-9995-e6a13fd44dc6] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:25:19.121581 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-c538ad5c-ccc8-443b-b4b3-11595d9df52e tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:25:19.122342 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-c538ad5c-ccc8-443b-b4b3-11595d9df52e tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:25:19.123115 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-c538ad5c-ccc8-443b-b4b3-11595d9df52e tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:25:19.123972 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-c538ad5c-ccc8-443b-b4b3-11595d9df52e tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:25:19.124422 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-c538ad5c-ccc8-443b-b4b3-11595d9df52e tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:25:19.124842 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-c538ad5c-ccc8-443b-b4b3-11595d9df52e tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:25:19.288366 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-c538ad5c-ccc8-443b-b4b3-11595d9df52e tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2e60cf2931f04f0e8e911adea32979dd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7cb3e3496c1a45c8942c3678c0c299e5', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:25:19.297107 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-c538ad5c-ccc8-443b-b4b3-11595d9df52e tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] 10.222.0.52 "GET /compute/v2.1/servers/e49199b7-e998-44fb-9995-e6a13fd44dc6" status: 200 len: 1313 microversion: 2.1 time: 0.192939 Aug 27 02:25:19.297918 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 181/364] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:25:19 2022] GET /compute/v2.1/servers/e49199b7-e998-44fb-9995-e6a13fd44dc6 => generated 1313 bytes in 195 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 27 02:25:19.750790 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-33be9821-31ae-416c-b79d-0eb3876f8245 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:25:19.751864 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-33be9821-31ae-416c-b79d-0eb3876f8245 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] [instance: 56dbfa6a-5b6f-4b82-977b-af735e6a6ac9] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:25:19.767294 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-33be9821-31ae-416c-b79d-0eb3876f8245 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:25:19.768253 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-33be9821-31ae-416c-b79d-0eb3876f8245 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:25:19.768824 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-33be9821-31ae-416c-b79d-0eb3876f8245 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:25:19.770297 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-33be9821-31ae-416c-b79d-0eb3876f8245 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:25:19.770759 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-33be9821-31ae-416c-b79d-0eb3876f8245 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:25:19.771191 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-33be9821-31ae-416c-b79d-0eb3876f8245 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:25:19.792391 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG dbcounter [-] [120863] Writing DB stats nova_cell1:SELECT=100 {{(pid=120863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 27 02:25:19.905737 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-33be9821-31ae-416c-b79d-0eb3876f8245 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:25:19.914122 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-33be9821-31ae-416c-b79d-0eb3876f8245 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:25:19.916060 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-33be9821-31ae-416c-b79d-0eb3876f8245 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:25:19.917580 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-33be9821-31ae-416c-b79d-0eb3876f8245 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] 10.222.0.52 "GET /compute/v2.1/servers/56dbfa6a-5b6f-4b82-977b-af735e6a6ac9" status: 200 len: 1407 microversion: 2.20 time: 0.171463 Aug 27 02:25:19.918506 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 184/365] 10.222.0.52 () {60 vars in 1277 bytes} [Sat Aug 27 02:25:19 2022] GET /compute/v2.1/servers/56dbfa6a-5b6f-4b82-977b-af735e6a6ac9 => generated 1407 bytes in 173 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 27 02:25:20.057542 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-32b3dc16-4c8e-4aea-ad42-d5ddea0b6c31 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Calling method '>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:25:20.058159 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-32b3dc16-4c8e-4aea-ad42-d5ddea0b6c31 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] [instance: e2bf7024-0779-4e08-844a-43b78187e770] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:25:20.064545 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-32b3dc16-4c8e-4aea-ad42-d5ddea0b6c31 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:25:20.065078 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-32b3dc16-4c8e-4aea-ad42-d5ddea0b6c31 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:25:20.065464 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-32b3dc16-4c8e-4aea-ad42-d5ddea0b6c31 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:25:20.066386 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-32b3dc16-4c8e-4aea-ad42-d5ddea0b6c31 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:25:20.066725 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-32b3dc16-4c8e-4aea-ad42-d5ddea0b6c31 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:25:20.067184 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-32b3dc16-4c8e-4aea-ad42-d5ddea0b6c31 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:25:20.207785 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-32b3dc16-4c8e-4aea-ad42-d5ddea0b6c31 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd945d512ae034ec08838d0a3afde2429', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '880ac4003248441c809a76cfd1b221eb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:25:20.215634 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-32b3dc16-4c8e-4aea-ad42-d5ddea0b6c31 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] 10.222.0.52 "GET /compute/v2.1/servers/e2bf7024-0779-4e08-844a-43b78187e770" status: 200 len: 1522 microversion: 2.1 time: 0.162861 Aug 27 02:25:20.216419 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 182/366] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:25:20 2022] GET /compute/v2.1/servers/e2bf7024-0779-4e08-844a-43b78187e770 => generated 1522 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:25:20.318793 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-85ed00ac-f4d1-43da-a7d2-77a309af071d tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:25:20.319426 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-85ed00ac-f4d1-43da-a7d2-77a309af071d tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] [instance: e49199b7-e998-44fb-9995-e6a13fd44dc6] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:25:20.325931 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-85ed00ac-f4d1-43da-a7d2-77a309af071d tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:25:20.326424 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-85ed00ac-f4d1-43da-a7d2-77a309af071d tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:25:20.326836 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-85ed00ac-f4d1-43da-a7d2-77a309af071d tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:25:20.327530 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-85ed00ac-f4d1-43da-a7d2-77a309af071d tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:25:20.327978 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-85ed00ac-f4d1-43da-a7d2-77a309af071d tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:25:20.328397 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-85ed00ac-f4d1-43da-a7d2-77a309af071d tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:25:20.506092 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-85ed00ac-f4d1-43da-a7d2-77a309af071d tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2e60cf2931f04f0e8e911adea32979dd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7cb3e3496c1a45c8942c3678c0c299e5', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:25:20.514398 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-85ed00ac-f4d1-43da-a7d2-77a309af071d tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] 10.222.0.52 "GET /compute/v2.1/servers/e49199b7-e998-44fb-9995-e6a13fd44dc6" status: 200 len: 1313 microversion: 2.1 time: 0.199851 Aug 27 02:25:20.514960 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 185/367] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:25:20 2022] GET /compute/v2.1/servers/e49199b7-e998-44fb-9995-e6a13fd44dc6 => generated 1313 bytes in 201 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:25:20.941768 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-fbcc5f15-1b94-47df-9ee8-b66e28337f88 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Calling method '>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:25:20.942107 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-fbcc5f15-1b94-47df-9ee8-b66e28337f88 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] [instance: 56dbfa6a-5b6f-4b82-977b-af735e6a6ac9] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:25:20.948416 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-fbcc5f15-1b94-47df-9ee8-b66e28337f88 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:25:20.949009 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-fbcc5f15-1b94-47df-9ee8-b66e28337f88 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:25:20.949566 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-fbcc5f15-1b94-47df-9ee8-b66e28337f88 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:25:20.950682 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-fbcc5f15-1b94-47df-9ee8-b66e28337f88 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:25:20.951267 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-fbcc5f15-1b94-47df-9ee8-b66e28337f88 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:25:20.952104 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-fbcc5f15-1b94-47df-9ee8-b66e28337f88 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:25:21.097065 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-fbcc5f15-1b94-47df-9ee8-b66e28337f88 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:25:21.101216 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG dbcounter [-] [120864] Writing DB stats nova_cell1:SELECT=100 {{(pid=120864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 27 02:25:21.107056 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-fbcc5f15-1b94-47df-9ee8-b66e28337f88 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:25:21.108611 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-fbcc5f15-1b94-47df-9ee8-b66e28337f88 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:25:21.109972 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-fbcc5f15-1b94-47df-9ee8-b66e28337f88 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] 10.222.0.52 "GET /compute/v2.1/servers/56dbfa6a-5b6f-4b82-977b-af735e6a6ac9" status: 200 len: 1407 microversion: 2.20 time: 0.172648 Aug 27 02:25:21.110559 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 183/368] 10.222.0.52 () {60 vars in 1277 bytes} [Sat Aug 27 02:25:20 2022] GET /compute/v2.1/servers/56dbfa6a-5b6f-4b82-977b-af735e6a6ac9 => generated 1407 bytes in 174 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 27 02:25:21.238572 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-42cde81f-2a1c-4ba4-aa60-22af3b704e67 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:25:21.239129 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-42cde81f-2a1c-4ba4-aa60-22af3b704e67 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] [instance: e2bf7024-0779-4e08-844a-43b78187e770] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:25:21.249592 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-42cde81f-2a1c-4ba4-aa60-22af3b704e67 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:25:21.250273 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-42cde81f-2a1c-4ba4-aa60-22af3b704e67 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:25:21.250722 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-42cde81f-2a1c-4ba4-aa60-22af3b704e67 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:25:21.253631 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-42cde81f-2a1c-4ba4-aa60-22af3b704e67 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:25:21.255385 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-42cde81f-2a1c-4ba4-aa60-22af3b704e67 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:25:21.256040 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-42cde81f-2a1c-4ba4-aa60-22af3b704e67 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:25:21.394880 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-42cde81f-2a1c-4ba4-aa60-22af3b704e67 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd945d512ae034ec08838d0a3afde2429', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '880ac4003248441c809a76cfd1b221eb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:25:21.403225 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-42cde81f-2a1c-4ba4-aa60-22af3b704e67 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] 10.222.0.52 "GET /compute/v2.1/servers/e2bf7024-0779-4e08-844a-43b78187e770" status: 200 len: 1522 microversion: 2.1 time: 0.168798 Aug 27 02:25:21.403794 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 186/369] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:25:21 2022] GET /compute/v2.1/servers/e2bf7024-0779-4e08-844a-43b78187e770 => generated 1522 bytes in 170 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:25:21.535725 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-e179ef3a-34e5-4bed-805f-25b9ed3f3fcc tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Calling method '>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:25:21.536399 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-e179ef3a-34e5-4bed-805f-25b9ed3f3fcc tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] [instance: e49199b7-e998-44fb-9995-e6a13fd44dc6] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:25:21.542740 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-e179ef3a-34e5-4bed-805f-25b9ed3f3fcc tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:25:21.543264 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-e179ef3a-34e5-4bed-805f-25b9ed3f3fcc tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:25:21.543691 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-e179ef3a-34e5-4bed-805f-25b9ed3f3fcc tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:25:21.544406 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-e179ef3a-34e5-4bed-805f-25b9ed3f3fcc tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:25:21.544847 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-e179ef3a-34e5-4bed-805f-25b9ed3f3fcc tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:25:21.545293 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-e179ef3a-34e5-4bed-805f-25b9ed3f3fcc tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:25:21.717100 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-e179ef3a-34e5-4bed-805f-25b9ed3f3fcc tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2e60cf2931f04f0e8e911adea32979dd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7cb3e3496c1a45c8942c3678c0c299e5', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:25:21.730916 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-e179ef3a-34e5-4bed-805f-25b9ed3f3fcc tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] 10.222.0.52 "GET /compute/v2.1/servers/e49199b7-e998-44fb-9995-e6a13fd44dc6" status: 200 len: 1313 microversion: 2.1 time: 0.199024 Aug 27 02:25:21.732196 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 184/370] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:25:21 2022] GET /compute/v2.1/servers/e49199b7-e998-44fb-9995-e6a13fd44dc6 => generated 1313 bytes in 201 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:25:22.135369 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-6d5f5f62-438c-429b-abfd-389e454035ec tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:25:22.136092 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-6d5f5f62-438c-429b-abfd-389e454035ec tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] [instance: 56dbfa6a-5b6f-4b82-977b-af735e6a6ac9] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:25:22.144749 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-6d5f5f62-438c-429b-abfd-389e454035ec tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:25:22.145271 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-6d5f5f62-438c-429b-abfd-389e454035ec tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:25:22.145707 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-6d5f5f62-438c-429b-abfd-389e454035ec tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:25:22.146403 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-6d5f5f62-438c-429b-abfd-389e454035ec tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:25:22.146859 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-6d5f5f62-438c-429b-abfd-389e454035ec tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:25:22.147281 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-6d5f5f62-438c-429b-abfd-389e454035ec tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:25:22.280835 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-6d5f5f62-438c-429b-abfd-389e454035ec tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:25:22.289905 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-6d5f5f62-438c-429b-abfd-389e454035ec tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:25:22.291272 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-6d5f5f62-438c-429b-abfd-389e454035ec tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:25:22.292368 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-6d5f5f62-438c-429b-abfd-389e454035ec tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] 10.222.0.52 "GET /compute/v2.1/servers/56dbfa6a-5b6f-4b82-977b-af735e6a6ac9" status: 200 len: 1407 microversion: 2.20 time: 0.163103 Aug 27 02:25:22.292916 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 187/371] 10.222.0.52 () {60 vars in 1277 bytes} [Sat Aug 27 02:25:22 2022] GET /compute/v2.1/servers/56dbfa6a-5b6f-4b82-977b-af735e6a6ac9 => generated 1407 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 27 02:25:22.424675 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-c5b0f4b3-0fda-472b-856d-df016a1cc1e0 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Calling method '>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:25:22.425284 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-c5b0f4b3-0fda-472b-856d-df016a1cc1e0 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] [instance: e2bf7024-0779-4e08-844a-43b78187e770] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:25:22.432137 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-c5b0f4b3-0fda-472b-856d-df016a1cc1e0 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:25:22.432617 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-c5b0f4b3-0fda-472b-856d-df016a1cc1e0 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:25:22.433026 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-c5b0f4b3-0fda-472b-856d-df016a1cc1e0 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:25:22.433744 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-c5b0f4b3-0fda-472b-856d-df016a1cc1e0 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:25:22.434198 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-c5b0f4b3-0fda-472b-856d-df016a1cc1e0 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:25:22.434628 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-c5b0f4b3-0fda-472b-856d-df016a1cc1e0 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:25:22.583538 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-c5b0f4b3-0fda-472b-856d-df016a1cc1e0 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd945d512ae034ec08838d0a3afde2429', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '880ac4003248441c809a76cfd1b221eb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:25:22.591830 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-c5b0f4b3-0fda-472b-856d-df016a1cc1e0 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] 10.222.0.52 "GET /compute/v2.1/servers/e2bf7024-0779-4e08-844a-43b78187e770" status: 200 len: 1522 microversion: 2.1 time: 0.171089 Aug 27 02:25:22.592382 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 185/372] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:25:22 2022] GET /compute/v2.1/servers/e2bf7024-0779-4e08-844a-43b78187e770 => generated 1522 bytes in 172 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:25:22.752399 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-ef914170-5574-4c48-a6e1-3e1313dd9231 tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:25:22.753006 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-ef914170-5574-4c48-a6e1-3e1313dd9231 tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] [instance: e49199b7-e998-44fb-9995-e6a13fd44dc6] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:25:22.759888 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-ef914170-5574-4c48-a6e1-3e1313dd9231 tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:25:22.760374 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-ef914170-5574-4c48-a6e1-3e1313dd9231 tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:25:22.760810 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-ef914170-5574-4c48-a6e1-3e1313dd9231 tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:25:22.761535 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-ef914170-5574-4c48-a6e1-3e1313dd9231 tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:25:22.761991 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-ef914170-5574-4c48-a6e1-3e1313dd9231 tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:25:22.762411 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-ef914170-5574-4c48-a6e1-3e1313dd9231 tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:25:22.908517 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-ef914170-5574-4c48-a6e1-3e1313dd9231 tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2e60cf2931f04f0e8e911adea32979dd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7cb3e3496c1a45c8942c3678c0c299e5', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:25:22.916417 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-ef914170-5574-4c48-a6e1-3e1313dd9231 tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] 10.222.0.52 "GET /compute/v2.1/servers/e49199b7-e998-44fb-9995-e6a13fd44dc6" status: 200 len: 1313 microversion: 2.1 time: 0.168026 Aug 27 02:25:22.916951 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 188/373] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:25:22 2022] GET /compute/v2.1/servers/e49199b7-e998-44fb-9995-e6a13fd44dc6 => generated 1313 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:25:23.317630 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-5caaa88d-f399-4eb7-b464-99bc335ab86f tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Calling method '>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:25:23.318293 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-5caaa88d-f399-4eb7-b464-99bc335ab86f tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] [instance: 56dbfa6a-5b6f-4b82-977b-af735e6a6ac9] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:25:23.324427 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-5caaa88d-f399-4eb7-b464-99bc335ab86f tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:25:23.325213 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-5caaa88d-f399-4eb7-b464-99bc335ab86f tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:25:23.325745 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-5caaa88d-f399-4eb7-b464-99bc335ab86f tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:25:23.327230 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-5caaa88d-f399-4eb7-b464-99bc335ab86f tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:25:23.327230 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-5caaa88d-f399-4eb7-b464-99bc335ab86f tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:25:23.327230 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-5caaa88d-f399-4eb7-b464-99bc335ab86f tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:25:23.462418 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-5caaa88d-f399-4eb7-b464-99bc335ab86f tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:25:23.470960 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-5caaa88d-f399-4eb7-b464-99bc335ab86f tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:25:23.472476 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-5caaa88d-f399-4eb7-b464-99bc335ab86f tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:25:23.473617 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-5caaa88d-f399-4eb7-b464-99bc335ab86f tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] 10.222.0.52 "GET /compute/v2.1/servers/56dbfa6a-5b6f-4b82-977b-af735e6a6ac9" status: 200 len: 1407 microversion: 2.20 time: 0.160656 Aug 27 02:25:23.474167 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 186/374] 10.222.0.52 () {60 vars in 1277 bytes} [Sat Aug 27 02:25:23 2022] GET /compute/v2.1/servers/56dbfa6a-5b6f-4b82-977b-af735e6a6ac9 => generated 1407 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 27 02:25:23.614005 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-0c8e4115-86d8-4c62-a05c-7976f98ef81b tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:25:23.614774 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-0c8e4115-86d8-4c62-a05c-7976f98ef81b tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] [instance: e2bf7024-0779-4e08-844a-43b78187e770] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:25:23.632038 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-0c8e4115-86d8-4c62-a05c-7976f98ef81b tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:25:23.632633 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-0c8e4115-86d8-4c62-a05c-7976f98ef81b tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:25:23.633204 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-0c8e4115-86d8-4c62-a05c-7976f98ef81b tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:25:23.634087 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-0c8e4115-86d8-4c62-a05c-7976f98ef81b tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:25:23.634783 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-0c8e4115-86d8-4c62-a05c-7976f98ef81b tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:25:23.635349 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-0c8e4115-86d8-4c62-a05c-7976f98ef81b tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:25:23.721254 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [req-503f2934-9352-45f7-bc2f-26727362092f req-be31cf39-bbfb-44d2-9759-d7f232608adb service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "e49199b7-e998-44fb-9995-e6a13fd44dc6", "name": "network-vif-plugged", "status": "completed", "tag": "55122ed4-7c38-4c0f-b4b9-a09312935cbc"}]} {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 27 02:25:23.731196 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [req-503f2934-9352-45f7-bc2f-26727362092f req-be31cf39-bbfb-44d2-9759-d7f232608adb service nova] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:25:23.732121 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [req-503f2934-9352-45f7-bc2f-26727362092f req-be31cf39-bbfb-44d2-9759-d7f232608adb service nova] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:25:23.732732 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [req-503f2934-9352-45f7-bc2f-26727362092f req-be31cf39-bbfb-44d2-9759-d7f232608adb service nova] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:25:23.748875 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.compute.server_external_events [req-503f2934-9352-45f7-bc2f-26727362092f req-be31cf39-bbfb-44d2-9759-d7f232608adb service nova] Creating event network-vif-plugged:55122ed4-7c38-4c0f-b4b9-a09312935cbc for instance e49199b7-e998-44fb-9995-e6a13fd44dc6 on hv-cinder-84630 Aug 27 02:25:23.754998 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [req-503f2934-9352-45f7-bc2f-26727362092f req-be31cf39-bbfb-44d2-9759-d7f232608adb service nova] 10.222.0.52 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.037954 Aug 27 02:25:23.755597 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 187/375] 10.222.0.52 () {64 vars in 1317 bytes} [Sat Aug 27 02:25:23 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 39 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 27 02:25:23.823740 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-0c8e4115-86d8-4c62-a05c-7976f98ef81b tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd945d512ae034ec08838d0a3afde2429', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '880ac4003248441c809a76cfd1b221eb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:25:23.833028 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-0c8e4115-86d8-4c62-a05c-7976f98ef81b tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] 10.222.0.52 "GET /compute/v2.1/servers/e2bf7024-0779-4e08-844a-43b78187e770" status: 200 len: 1522 microversion: 2.1 time: 0.223759 Aug 27 02:25:23.844716 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 189/376] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:25:23 2022] GET /compute/v2.1/servers/e2bf7024-0779-4e08-844a-43b78187e770 => generated 1522 bytes in 236 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:25:23.938115 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-0fad2303-e803-45b3-8b61-5856d5c5dd3d tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Calling method '>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:25:23.938793 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-0fad2303-e803-45b3-8b61-5856d5c5dd3d tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] [instance: e49199b7-e998-44fb-9995-e6a13fd44dc6] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:25:23.945591 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-0fad2303-e803-45b3-8b61-5856d5c5dd3d tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:25:23.946067 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-0fad2303-e803-45b3-8b61-5856d5c5dd3d tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:25:23.946480 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-0fad2303-e803-45b3-8b61-5856d5c5dd3d tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:25:23.947160 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-0fad2303-e803-45b3-8b61-5856d5c5dd3d tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:25:23.947768 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-0fad2303-e803-45b3-8b61-5856d5c5dd3d tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:25:23.948201 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-0fad2303-e803-45b3-8b61-5856d5c5dd3d tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:25:24.116197 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-0fad2303-e803-45b3-8b61-5856d5c5dd3d tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2e60cf2931f04f0e8e911adea32979dd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7cb3e3496c1a45c8942c3678c0c299e5', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:25:24.129008 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-0fad2303-e803-45b3-8b61-5856d5c5dd3d tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] 10.222.0.52 "GET /compute/v2.1/servers/e49199b7-e998-44fb-9995-e6a13fd44dc6" status: 200 len: 1313 microversion: 2.1 time: 0.194108 Aug 27 02:25:24.130182 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 188/377] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:25:23 2022] GET /compute/v2.1/servers/e49199b7-e998-44fb-9995-e6a13fd44dc6 => generated 1313 bytes in 197 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:25:24.497994 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-e775d552-6848-4b63-9963-7396691c6031 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:25:24.499042 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-e775d552-6848-4b63-9963-7396691c6031 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] [instance: 56dbfa6a-5b6f-4b82-977b-af735e6a6ac9] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:25:24.503287 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG dbcounter [-] [120863] Writing DB stats nova_api:SELECT=80 {{(pid=120863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 27 02:25:24.512756 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-e775d552-6848-4b63-9963-7396691c6031 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:25:24.513417 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-e775d552-6848-4b63-9963-7396691c6031 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:25:24.513697 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-e775d552-6848-4b63-9963-7396691c6031 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:25:24.514401 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-e775d552-6848-4b63-9963-7396691c6031 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:25:24.514856 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-e775d552-6848-4b63-9963-7396691c6031 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:25:24.515277 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-e775d552-6848-4b63-9963-7396691c6031 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:25:24.723650 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-e775d552-6848-4b63-9963-7396691c6031 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:25:24.732903 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-e775d552-6848-4b63-9963-7396691c6031 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:25:24.734668 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-e775d552-6848-4b63-9963-7396691c6031 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:25:24.735898 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-e775d552-6848-4b63-9963-7396691c6031 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] 10.222.0.52 "GET /compute/v2.1/servers/56dbfa6a-5b6f-4b82-977b-af735e6a6ac9" status: 200 len: 1407 microversion: 2.20 time: 0.243832 Aug 27 02:25:24.736469 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 190/378] 10.222.0.52 () {60 vars in 1277 bytes} [Sat Aug 27 02:25:24 2022] GET /compute/v2.1/servers/56dbfa6a-5b6f-4b82-977b-af735e6a6ac9 => generated 1407 bytes in 245 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 27 02:25:24.861681 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-8a27e147-493f-4d41-8c2b-01de5f01be10 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Calling method '>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:25:24.862278 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-8a27e147-493f-4d41-8c2b-01de5f01be10 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] [instance: e2bf7024-0779-4e08-844a-43b78187e770] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:25:24.869463 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-8a27e147-493f-4d41-8c2b-01de5f01be10 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:25:24.869942 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-8a27e147-493f-4d41-8c2b-01de5f01be10 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:25:24.870364 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-8a27e147-493f-4d41-8c2b-01de5f01be10 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:25:24.871043 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-8a27e147-493f-4d41-8c2b-01de5f01be10 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:25:24.871512 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-8a27e147-493f-4d41-8c2b-01de5f01be10 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:25:24.872026 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-8a27e147-493f-4d41-8c2b-01de5f01be10 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:25:25.020753 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-8a27e147-493f-4d41-8c2b-01de5f01be10 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd945d512ae034ec08838d0a3afde2429', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '880ac4003248441c809a76cfd1b221eb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:25:25.028815 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-8a27e147-493f-4d41-8c2b-01de5f01be10 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] 10.222.0.52 "GET /compute/v2.1/servers/e2bf7024-0779-4e08-844a-43b78187e770" status: 200 len: 1522 microversion: 2.1 time: 0.178415 Aug 27 02:25:25.029546 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 189/379] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:25:24 2022] GET /compute/v2.1/servers/e2bf7024-0779-4e08-844a-43b78187e770 => generated 1522 bytes in 180 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:25:25.152111 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-48a75922-22cd-4d54-96d7-07e5a62eb1fe tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:25:25.152775 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-48a75922-22cd-4d54-96d7-07e5a62eb1fe tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] [instance: e49199b7-e998-44fb-9995-e6a13fd44dc6] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:25:25.159102 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-48a75922-22cd-4d54-96d7-07e5a62eb1fe tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:25:25.159874 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-48a75922-22cd-4d54-96d7-07e5a62eb1fe tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:25:25.160333 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-48a75922-22cd-4d54-96d7-07e5a62eb1fe tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:25:25.161036 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-48a75922-22cd-4d54-96d7-07e5a62eb1fe tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:25:25.161557 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-48a75922-22cd-4d54-96d7-07e5a62eb1fe tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:25:25.161987 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-48a75922-22cd-4d54-96d7-07e5a62eb1fe tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:25:25.309557 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-48a75922-22cd-4d54-96d7-07e5a62eb1fe tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2e60cf2931f04f0e8e911adea32979dd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7cb3e3496c1a45c8942c3678c0c299e5', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:25:25.318848 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-48a75922-22cd-4d54-96d7-07e5a62eb1fe tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] 10.222.0.52 "GET /compute/v2.1/servers/e49199b7-e998-44fb-9995-e6a13fd44dc6" status: 200 len: 1313 microversion: 2.1 time: 0.170471 Aug 27 02:25:25.318848 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 191/380] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:25:25 2022] GET /compute/v2.1/servers/e49199b7-e998-44fb-9995-e6a13fd44dc6 => generated 1313 bytes in 172 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:25:25.759787 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-649a34b9-e196-4648-a455-5b986731a8c8 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Calling method '>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:25:25.760392 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-649a34b9-e196-4648-a455-5b986731a8c8 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] [instance: 56dbfa6a-5b6f-4b82-977b-af735e6a6ac9] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:25:25.766831 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-649a34b9-e196-4648-a455-5b986731a8c8 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:25:25.767318 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-649a34b9-e196-4648-a455-5b986731a8c8 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:25:25.767809 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-649a34b9-e196-4648-a455-5b986731a8c8 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:25:25.768498 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-649a34b9-e196-4648-a455-5b986731a8c8 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:25:25.768946 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-649a34b9-e196-4648-a455-5b986731a8c8 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:25:25.769677 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-649a34b9-e196-4648-a455-5b986731a8c8 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:25:25.920511 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-649a34b9-e196-4648-a455-5b986731a8c8 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:25:25.929547 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-649a34b9-e196-4648-a455-5b986731a8c8 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:25:25.930978 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-649a34b9-e196-4648-a455-5b986731a8c8 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:25:25.932071 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-649a34b9-e196-4648-a455-5b986731a8c8 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] 10.222.0.52 "GET /compute/v2.1/servers/56dbfa6a-5b6f-4b82-977b-af735e6a6ac9" status: 200 len: 1407 microversion: 2.20 time: 0.176660 Aug 27 02:25:25.932627 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 190/381] 10.222.0.52 () {60 vars in 1277 bytes} [Sat Aug 27 02:25:25 2022] GET /compute/v2.1/servers/56dbfa6a-5b6f-4b82-977b-af735e6a6ac9 => generated 1407 bytes in 178 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 27 02:25:26.050273 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-91e20e9c-8c9f-4035-b2c0-41888c8fa4f7 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:25:26.050863 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-91e20e9c-8c9f-4035-b2c0-41888c8fa4f7 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] [instance: e2bf7024-0779-4e08-844a-43b78187e770] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:25:26.057941 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-91e20e9c-8c9f-4035-b2c0-41888c8fa4f7 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:25:26.058428 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-91e20e9c-8c9f-4035-b2c0-41888c8fa4f7 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:25:26.058876 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-91e20e9c-8c9f-4035-b2c0-41888c8fa4f7 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:25:26.059753 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-91e20e9c-8c9f-4035-b2c0-41888c8fa4f7 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:25:26.060204 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-91e20e9c-8c9f-4035-b2c0-41888c8fa4f7 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:25:26.060623 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-91e20e9c-8c9f-4035-b2c0-41888c8fa4f7 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:25:26.214335 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-91e20e9c-8c9f-4035-b2c0-41888c8fa4f7 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd945d512ae034ec08838d0a3afde2429', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '880ac4003248441c809a76cfd1b221eb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:25:26.222984 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-91e20e9c-8c9f-4035-b2c0-41888c8fa4f7 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] 10.222.0.52 "GET /compute/v2.1/servers/e2bf7024-0779-4e08-844a-43b78187e770" status: 200 len: 1522 microversion: 2.1 time: 0.176603 Aug 27 02:25:26.223563 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 192/382] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:25:26 2022] GET /compute/v2.1/servers/e2bf7024-0779-4e08-844a-43b78187e770 => generated 1522 bytes in 178 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:25:26.344073 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-168df57a-6905-49c3-9f23-2c2900df395c tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Calling method '>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:25:26.344672 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-168df57a-6905-49c3-9f23-2c2900df395c tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] [instance: e49199b7-e998-44fb-9995-e6a13fd44dc6] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:25:26.352321 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-168df57a-6905-49c3-9f23-2c2900df395c tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:25:26.352812 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-168df57a-6905-49c3-9f23-2c2900df395c tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:25:26.353271 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-168df57a-6905-49c3-9f23-2c2900df395c tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:25:26.354002 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-168df57a-6905-49c3-9f23-2c2900df395c tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:25:26.354448 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-168df57a-6905-49c3-9f23-2c2900df395c tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:25:26.354862 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-168df57a-6905-49c3-9f23-2c2900df395c tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:25:26.494051 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-168df57a-6905-49c3-9f23-2c2900df395c tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2e60cf2931f04f0e8e911adea32979dd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7cb3e3496c1a45c8942c3678c0c299e5', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:25:26.507936 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-168df57a-6905-49c3-9f23-2c2900df395c tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] 10.222.0.52 "GET /compute/v2.1/servers/e49199b7-e998-44fb-9995-e6a13fd44dc6" status: 200 len: 1313 microversion: 2.1 time: 0.167524 Aug 27 02:25:26.509225 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 191/383] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:25:26 2022] GET /compute/v2.1/servers/e49199b7-e998-44fb-9995-e6a13fd44dc6 => generated 1313 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:25:26.956040 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-6d3c470f-4d38-47e3-b847-65274781e001 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:25:26.956705 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-6d3c470f-4d38-47e3-b847-65274781e001 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] [instance: 56dbfa6a-5b6f-4b82-977b-af735e6a6ac9] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:25:26.962888 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-6d3c470f-4d38-47e3-b847-65274781e001 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:25:26.963559 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-6d3c470f-4d38-47e3-b847-65274781e001 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:25:26.964305 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-6d3c470f-4d38-47e3-b847-65274781e001 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:25:26.964998 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-6d3c470f-4d38-47e3-b847-65274781e001 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:25:26.965459 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-6d3c470f-4d38-47e3-b847-65274781e001 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:25:26.965876 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-6d3c470f-4d38-47e3-b847-65274781e001 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:25:27.131572 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-6d3c470f-4d38-47e3-b847-65274781e001 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:25:27.140410 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-6d3c470f-4d38-47e3-b847-65274781e001 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:25:27.143544 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-6d3c470f-4d38-47e3-b847-65274781e001 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:25:27.145023 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-6d3c470f-4d38-47e3-b847-65274781e001 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] 10.222.0.52 "GET /compute/v2.1/servers/56dbfa6a-5b6f-4b82-977b-af735e6a6ac9" status: 200 len: 1599 microversion: 2.20 time: 0.192924 Aug 27 02:25:27.145629 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 193/384] 10.222.0.52 () {60 vars in 1277 bytes} [Sat Aug 27 02:25:26 2022] GET /compute/v2.1/servers/56dbfa6a-5b6f-4b82-977b-af735e6a6ac9 => generated 1599 bytes in 194 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 27 02:25:27.176450 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-465ff5ba-a330-4efc-9624-baf4ad4d62d8 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Calling method '>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:25:27.177032 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-465ff5ba-a330-4efc-9624-baf4ad4d62d8 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] [instance: 56dbfa6a-5b6f-4b82-977b-af735e6a6ac9] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:25:27.183090 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-465ff5ba-a330-4efc-9624-baf4ad4d62d8 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:25:27.183611 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-465ff5ba-a330-4efc-9624-baf4ad4d62d8 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:25:27.184029 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-465ff5ba-a330-4efc-9624-baf4ad4d62d8 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:25:27.184661 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-465ff5ba-a330-4efc-9624-baf4ad4d62d8 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:25:27.185105 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-465ff5ba-a330-4efc-9624-baf4ad4d62d8 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:25:27.185548 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-465ff5ba-a330-4efc-9624-baf4ad4d62d8 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:25:27.244464 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-4699d08b-4bbe-499e-929a-e9b3e70cff79 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:25:27.245215 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-4699d08b-4bbe-499e-929a-e9b3e70cff79 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] [instance: e2bf7024-0779-4e08-844a-43b78187e770] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:25:27.251447 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-4699d08b-4bbe-499e-929a-e9b3e70cff79 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:25:27.251945 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-4699d08b-4bbe-499e-929a-e9b3e70cff79 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:25:27.252382 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-4699d08b-4bbe-499e-929a-e9b3e70cff79 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:25:27.253083 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-4699d08b-4bbe-499e-929a-e9b3e70cff79 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:25:27.253640 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-4699d08b-4bbe-499e-929a-e9b3e70cff79 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:25:27.254344 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-4699d08b-4bbe-499e-929a-e9b3e70cff79 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:25:27.333937 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-465ff5ba-a330-4efc-9624-baf4ad4d62d8 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] 10.222.0.52 "GET /compute/v2.1/servers/56dbfa6a-5b6f-4b82-977b-af735e6a6ac9/os-interface" status: 200 len: 284 microversion: 2.20 time: 0.160972 Aug 27 02:25:27.335326 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 192/385] 10.222.0.52 () {60 vars in 1316 bytes} [Sat Aug 27 02:25:27 2022] GET /compute/v2.1/servers/56dbfa6a-5b6f-4b82-977b-af735e6a6ac9/os-interface => generated 284 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 359 bytes (2 switches on core 0) Aug 27 02:25:27.529161 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-4699d08b-4bbe-499e-929a-e9b3e70cff79 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd945d512ae034ec08838d0a3afde2429', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '880ac4003248441c809a76cfd1b221eb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:25:27.536410 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-d7586462-e1e4-4232-9656-1cec7a528228 tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Calling method '>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:25:27.537075 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-d7586462-e1e4-4232-9656-1cec7a528228 tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] [instance: e49199b7-e998-44fb-9995-e6a13fd44dc6] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:25:27.550938 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-d7586462-e1e4-4232-9656-1cec7a528228 tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:25:27.551838 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-4699d08b-4bbe-499e-929a-e9b3e70cff79 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] 10.222.0.52 "GET /compute/v2.1/servers/e2bf7024-0779-4e08-844a-43b78187e770" status: 200 len: 1522 microversion: 2.1 time: 0.311453 Aug 27 02:25:27.552899 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 194/386] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:25:27 2022] GET /compute/v2.1/servers/e2bf7024-0779-4e08-844a-43b78187e770 => generated 1522 bytes in 313 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:25:27.554030 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-d7586462-e1e4-4232-9656-1cec7a528228 tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:25:27.554628 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-d7586462-e1e4-4232-9656-1cec7a528228 tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:25:27.555686 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-d7586462-e1e4-4232-9656-1cec7a528228 tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:25:27.556293 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-d7586462-e1e4-4232-9656-1cec7a528228 tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:25:27.557075 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-d7586462-e1e4-4232-9656-1cec7a528228 tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:25:27.821837 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-d7586462-e1e4-4232-9656-1cec7a528228 tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2e60cf2931f04f0e8e911adea32979dd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7cb3e3496c1a45c8942c3678c0c299e5', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:25:27.835426 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-d7586462-e1e4-4232-9656-1cec7a528228 tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] 10.222.0.52 "GET /compute/v2.1/servers/e49199b7-e998-44fb-9995-e6a13fd44dc6" status: 200 len: 1313 microversion: 2.1 time: 0.303489 Aug 27 02:25:27.836306 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 193/387] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:25:27 2022] GET /compute/v2.1/servers/e49199b7-e998-44fb-9995-e6a13fd44dc6 => generated 1313 bytes in 305 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:25:28.573920 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-086c1ae5-a450-474a-b826-e8cf53c89324 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:25:28.574550 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-086c1ae5-a450-474a-b826-e8cf53c89324 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] [instance: e2bf7024-0779-4e08-844a-43b78187e770] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:25:28.581416 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-086c1ae5-a450-474a-b826-e8cf53c89324 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:25:28.581975 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-086c1ae5-a450-474a-b826-e8cf53c89324 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:25:28.582575 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-086c1ae5-a450-474a-b826-e8cf53c89324 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:25:28.583124 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-086c1ae5-a450-474a-b826-e8cf53c89324 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:25:28.583572 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-086c1ae5-a450-474a-b826-e8cf53c89324 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:25:28.584010 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-086c1ae5-a450-474a-b826-e8cf53c89324 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:25:28.738814 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [req-e06492ce-9e44-4d0e-89b9-213aafe8af72 req-db921340-e8d7-4c85-952e-130a9c734a9d service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "56dbfa6a-5b6f-4b82-977b-af735e6a6ac9", "tag": "520bd5fb-723a-4570-83ca-72759c588707"}]} {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 27 02:25:28.754108 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [req-e06492ce-9e44-4d0e-89b9-213aafe8af72 req-db921340-e8d7-4c85-952e-130a9c734a9d service nova] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:25:28.754770 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [req-e06492ce-9e44-4d0e-89b9-213aafe8af72 req-db921340-e8d7-4c85-952e-130a9c734a9d service nova] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:25:28.755260 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [req-e06492ce-9e44-4d0e-89b9-213aafe8af72 req-db921340-e8d7-4c85-952e-130a9c734a9d service nova] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:25:28.767337 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.compute.server_external_events [req-e06492ce-9e44-4d0e-89b9-213aafe8af72 req-db921340-e8d7-4c85-952e-130a9c734a9d service nova] Creating event network-changed:520bd5fb-723a-4570-83ca-72759c588707 for instance 56dbfa6a-5b6f-4b82-977b-af735e6a6ac9 on hv-cinder-84630 Aug 27 02:25:28.773524 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [req-e06492ce-9e44-4d0e-89b9-213aafe8af72 req-db921340-e8d7-4c85-952e-130a9c734a9d service nova] 10.222.0.52 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.040695 Aug 27 02:25:28.773907 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-086c1ae5-a450-474a-b826-e8cf53c89324 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd945d512ae034ec08838d0a3afde2429', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '880ac4003248441c809a76cfd1b221eb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:25:28.774352 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 194/388] 10.222.0.52 () {64 vars in 1317 bytes} [Sat Aug 27 02:25:28 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 42 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 27 02:25:28.782187 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-086c1ae5-a450-474a-b826-e8cf53c89324 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] 10.222.0.52 "GET /compute/v2.1/servers/e2bf7024-0779-4e08-844a-43b78187e770" status: 200 len: 1522 microversion: 2.1 time: 0.212139 Aug 27 02:25:28.783057 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 195/389] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:25:28 2022] GET /compute/v2.1/servers/e2bf7024-0779-4e08-844a-43b78187e770 => generated 1522 bytes in 214 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 27 02:25:28.866099 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-3a3ccd8b-6c9a-4ff2-b985-72126e303b04 tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Calling method '>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:25:28.866712 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-3a3ccd8b-6c9a-4ff2-b985-72126e303b04 tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] [instance: e49199b7-e998-44fb-9995-e6a13fd44dc6] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:25:28.873714 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-3a3ccd8b-6c9a-4ff2-b985-72126e303b04 tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:25:28.874205 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-3a3ccd8b-6c9a-4ff2-b985-72126e303b04 tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:25:28.874636 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-3a3ccd8b-6c9a-4ff2-b985-72126e303b04 tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:25:28.875312 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-3a3ccd8b-6c9a-4ff2-b985-72126e303b04 tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:25:28.875754 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-3a3ccd8b-6c9a-4ff2-b985-72126e303b04 tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:25:28.876169 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-3a3ccd8b-6c9a-4ff2-b985-72126e303b04 tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:25:29.023016 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-3a3ccd8b-6c9a-4ff2-b985-72126e303b04 tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2e60cf2931f04f0e8e911adea32979dd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7cb3e3496c1a45c8942c3678c0c299e5', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:25:29.030780 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-3a3ccd8b-6c9a-4ff2-b985-72126e303b04 tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] 10.222.0.52 "GET /compute/v2.1/servers/e49199b7-e998-44fb-9995-e6a13fd44dc6" status: 200 len: 1313 microversion: 2.1 time: 0.169780 Aug 27 02:25:29.031325 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 195/390] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:25:28 2022] GET /compute/v2.1/servers/e49199b7-e998-44fb-9995-e6a13fd44dc6 => generated 1313 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:25:29.804200 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-c738f367-917f-4120-a86e-032c30b06e38 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:25:29.804803 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-c738f367-917f-4120-a86e-032c30b06e38 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] [instance: e2bf7024-0779-4e08-844a-43b78187e770] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:25:29.811362 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-c738f367-917f-4120-a86e-032c30b06e38 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:25:29.811851 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-c738f367-917f-4120-a86e-032c30b06e38 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:25:29.812261 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-c738f367-917f-4120-a86e-032c30b06e38 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:25:29.812958 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-c738f367-917f-4120-a86e-032c30b06e38 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:25:29.813443 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-c738f367-917f-4120-a86e-032c30b06e38 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:25:29.813868 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-c738f367-917f-4120-a86e-032c30b06e38 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:25:29.954421 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-c738f367-917f-4120-a86e-032c30b06e38 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd945d512ae034ec08838d0a3afde2429', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '880ac4003248441c809a76cfd1b221eb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:25:29.962547 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-c738f367-917f-4120-a86e-032c30b06e38 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] 10.222.0.52 "GET /compute/v2.1/servers/e2bf7024-0779-4e08-844a-43b78187e770" status: 200 len: 1522 microversion: 2.1 time: 0.163012 Aug 27 02:25:29.963186 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 196/391] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:25:29 2022] GET /compute/v2.1/servers/e2bf7024-0779-4e08-844a-43b78187e770 => generated 1522 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:25:30.051930 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-29c4e12e-5d07-4de5-9a5a-09430bb5cc31 tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Calling method '>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:25:30.052574 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-29c4e12e-5d07-4de5-9a5a-09430bb5cc31 tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] [instance: e49199b7-e998-44fb-9995-e6a13fd44dc6] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:25:30.058655 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-29c4e12e-5d07-4de5-9a5a-09430bb5cc31 tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:25:30.059100 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-29c4e12e-5d07-4de5-9a5a-09430bb5cc31 tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:25:30.059671 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-29c4e12e-5d07-4de5-9a5a-09430bb5cc31 tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:25:30.060413 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-29c4e12e-5d07-4de5-9a5a-09430bb5cc31 tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:25:30.060857 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-29c4e12e-5d07-4de5-9a5a-09430bb5cc31 tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:25:30.061310 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-29c4e12e-5d07-4de5-9a5a-09430bb5cc31 tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:25:30.205092 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-29c4e12e-5d07-4de5-9a5a-09430bb5cc31 tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2e60cf2931f04f0e8e911adea32979dd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7cb3e3496c1a45c8942c3678c0c299e5', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:25:30.213756 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-29c4e12e-5d07-4de5-9a5a-09430bb5cc31 tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] 10.222.0.52 "GET /compute/v2.1/servers/e49199b7-e998-44fb-9995-e6a13fd44dc6" status: 200 len: 1313 microversion: 2.1 time: 0.165784 Aug 27 02:25:30.214331 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 196/392] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:25:30 2022] GET /compute/v2.1/servers/e49199b7-e998-44fb-9995-e6a13fd44dc6 => generated 1313 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:25:30.985951 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-05ec4300-5631-452c-a0a5-9390cf1c3741 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:25:30.986514 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-05ec4300-5631-452c-a0a5-9390cf1c3741 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] [instance: e2bf7024-0779-4e08-844a-43b78187e770] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:25:30.993475 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-05ec4300-5631-452c-a0a5-9390cf1c3741 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:25:30.993977 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-05ec4300-5631-452c-a0a5-9390cf1c3741 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:25:30.994400 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-05ec4300-5631-452c-a0a5-9390cf1c3741 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:25:30.995092 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-05ec4300-5631-452c-a0a5-9390cf1c3741 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:25:30.995539 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-05ec4300-5631-452c-a0a5-9390cf1c3741 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:25:30.995958 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-05ec4300-5631-452c-a0a5-9390cf1c3741 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:25:31.137479 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-05ec4300-5631-452c-a0a5-9390cf1c3741 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd945d512ae034ec08838d0a3afde2429', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '880ac4003248441c809a76cfd1b221eb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:25:31.146086 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-05ec4300-5631-452c-a0a5-9390cf1c3741 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] 10.222.0.52 "GET /compute/v2.1/servers/e2bf7024-0779-4e08-844a-43b78187e770" status: 200 len: 1522 microversion: 2.1 time: 0.164351 Aug 27 02:25:31.146655 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 197/393] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:25:30 2022] GET /compute/v2.1/servers/e2bf7024-0779-4e08-844a-43b78187e770 => generated 1522 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:25:31.234506 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-a5f73334-e274-46ac-b44e-9d060361c101 tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Calling method '>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:25:31.235078 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-a5f73334-e274-46ac-b44e-9d060361c101 tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] [instance: e49199b7-e998-44fb-9995-e6a13fd44dc6] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:25:31.241860 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-a5f73334-e274-46ac-b44e-9d060361c101 tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:25:31.242254 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-a5f73334-e274-46ac-b44e-9d060361c101 tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:25:31.242685 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-a5f73334-e274-46ac-b44e-9d060361c101 tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:25:31.243387 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-a5f73334-e274-46ac-b44e-9d060361c101 tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:25:31.243827 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-a5f73334-e274-46ac-b44e-9d060361c101 tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:25:31.244223 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-a5f73334-e274-46ac-b44e-9d060361c101 tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:25:31.377820 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-a5f73334-e274-46ac-b44e-9d060361c101 tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2e60cf2931f04f0e8e911adea32979dd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7cb3e3496c1a45c8942c3678c0c299e5', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:25:31.385948 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-a5f73334-e274-46ac-b44e-9d060361c101 tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] 10.222.0.52 "GET /compute/v2.1/servers/e49199b7-e998-44fb-9995-e6a13fd44dc6" status: 200 len: 1313 microversion: 2.1 time: 0.155295 Aug 27 02:25:31.386599 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 197/394] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:25:31 2022] GET /compute/v2.1/servers/e49199b7-e998-44fb-9995-e6a13fd44dc6 => generated 1313 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:25:32.171549 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-6dcfca60-c3bc-4f11-b8c3-36bf1bd4c9c2 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:25:32.172538 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-6dcfca60-c3bc-4f11-b8c3-36bf1bd4c9c2 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] [instance: e2bf7024-0779-4e08-844a-43b78187e770] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:25:32.183721 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-6dcfca60-c3bc-4f11-b8c3-36bf1bd4c9c2 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:25:32.184538 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-6dcfca60-c3bc-4f11-b8c3-36bf1bd4c9c2 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:25:32.185330 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-6dcfca60-c3bc-4f11-b8c3-36bf1bd4c9c2 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:25:32.186705 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-6dcfca60-c3bc-4f11-b8c3-36bf1bd4c9c2 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:25:32.187450 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-6dcfca60-c3bc-4f11-b8c3-36bf1bd4c9c2 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:25:32.188148 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-6dcfca60-c3bc-4f11-b8c3-36bf1bd4c9c2 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:25:32.343272 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-6dcfca60-c3bc-4f11-b8c3-36bf1bd4c9c2 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd945d512ae034ec08838d0a3afde2429', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '880ac4003248441c809a76cfd1b221eb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:25:32.352846 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-6dcfca60-c3bc-4f11-b8c3-36bf1bd4c9c2 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] 10.222.0.52 "GET /compute/v2.1/servers/e2bf7024-0779-4e08-844a-43b78187e770" status: 200 len: 1522 microversion: 2.1 time: 0.187837 Aug 27 02:25:32.353448 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 198/395] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:25:32 2022] GET /compute/v2.1/servers/e2bf7024-0779-4e08-844a-43b78187e770 => generated 1522 bytes in 189 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:25:32.409811 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-ab99a714-ecca-4962-b710-884a7a351ed5 tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Calling method '>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:25:32.410929 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-ab99a714-ecca-4962-b710-884a7a351ed5 tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] [instance: e49199b7-e998-44fb-9995-e6a13fd44dc6] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:25:32.413792 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG dbcounter [-] [120864] Writing DB stats nova_api:SELECT=80 {{(pid=120864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 27 02:25:32.421288 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-ab99a714-ecca-4962-b710-884a7a351ed5 tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:25:32.421966 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-ab99a714-ecca-4962-b710-884a7a351ed5 tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:25:32.422492 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-ab99a714-ecca-4962-b710-884a7a351ed5 tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:25:32.423433 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-ab99a714-ecca-4962-b710-884a7a351ed5 tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:25:32.424036 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-ab99a714-ecca-4962-b710-884a7a351ed5 tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:25:32.424615 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-ab99a714-ecca-4962-b710-884a7a351ed5 tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:25:32.557437 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-ab99a714-ecca-4962-b710-884a7a351ed5 tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2e60cf2931f04f0e8e911adea32979dd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7cb3e3496c1a45c8942c3678c0c299e5', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:25:32.565664 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-ab99a714-ecca-4962-b710-884a7a351ed5 tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] 10.222.0.52 "GET /compute/v2.1/servers/e49199b7-e998-44fb-9995-e6a13fd44dc6" status: 200 len: 1313 microversion: 2.1 time: 0.161548 Aug 27 02:25:32.566222 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 198/396] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:25:32 2022] GET /compute/v2.1/servers/e49199b7-e998-44fb-9995-e6a13fd44dc6 => generated 1313 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:25:33.375691 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-5a5c141e-304b-4f3e-8162-ec8c0304f6c4 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:25:33.376327 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-5a5c141e-304b-4f3e-8162-ec8c0304f6c4 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] [instance: e2bf7024-0779-4e08-844a-43b78187e770] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:25:33.383446 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-5a5c141e-304b-4f3e-8162-ec8c0304f6c4 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:25:33.390651 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-5a5c141e-304b-4f3e-8162-ec8c0304f6c4 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.007s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:25:33.399264 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-5a5c141e-304b-4f3e-8162-ec8c0304f6c4 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.009s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:25:33.400048 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-5a5c141e-304b-4f3e-8162-ec8c0304f6c4 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:25:33.400520 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-5a5c141e-304b-4f3e-8162-ec8c0304f6c4 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:25:33.400998 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-5a5c141e-304b-4f3e-8162-ec8c0304f6c4 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:25:33.537563 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-5a5c141e-304b-4f3e-8162-ec8c0304f6c4 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd945d512ae034ec08838d0a3afde2429', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '880ac4003248441c809a76cfd1b221eb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:25:33.546074 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-5a5c141e-304b-4f3e-8162-ec8c0304f6c4 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] 10.222.0.52 "GET /compute/v2.1/servers/e2bf7024-0779-4e08-844a-43b78187e770" status: 200 len: 1522 microversion: 2.1 time: 0.174474 Aug 27 02:25:33.546631 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 199/397] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:25:33 2022] GET /compute/v2.1/servers/e2bf7024-0779-4e08-844a-43b78187e770 => generated 1522 bytes in 176 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:25:33.587168 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-8ab2dc9b-ab67-4ecc-ae39-da958de320f9 tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Calling method '>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:25:33.587735 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-8ab2dc9b-ab67-4ecc-ae39-da958de320f9 tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] [instance: e49199b7-e998-44fb-9995-e6a13fd44dc6] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:25:33.594070 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-8ab2dc9b-ab67-4ecc-ae39-da958de320f9 tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:25:33.594551 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-8ab2dc9b-ab67-4ecc-ae39-da958de320f9 tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:25:33.594977 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-8ab2dc9b-ab67-4ecc-ae39-da958de320f9 tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:25:33.595655 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-8ab2dc9b-ab67-4ecc-ae39-da958de320f9 tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:25:33.596106 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-8ab2dc9b-ab67-4ecc-ae39-da958de320f9 tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:25:33.596514 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-8ab2dc9b-ab67-4ecc-ae39-da958de320f9 tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:25:33.743847 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-8ab2dc9b-ab67-4ecc-ae39-da958de320f9 tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2e60cf2931f04f0e8e911adea32979dd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7cb3e3496c1a45c8942c3678c0c299e5', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:25:33.755441 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-8ab2dc9b-ab67-4ecc-ae39-da958de320f9 tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] 10.222.0.52 "GET /compute/v2.1/servers/e49199b7-e998-44fb-9995-e6a13fd44dc6" status: 200 len: 1495 microversion: 2.1 time: 0.172120 Aug 27 02:25:33.756177 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 199/398] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:25:33 2022] GET /compute/v2.1/servers/e49199b7-e998-44fb-9995-e6a13fd44dc6 => generated 1495 bytes in 173 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:25:34.569597 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-9be6b8ff-46a4-4a2a-8a1d-6748051e7f34 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:25:34.570247 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-9be6b8ff-46a4-4a2a-8a1d-6748051e7f34 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] [instance: e2bf7024-0779-4e08-844a-43b78187e770] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:25:34.577607 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-9be6b8ff-46a4-4a2a-8a1d-6748051e7f34 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:25:34.578422 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-9be6b8ff-46a4-4a2a-8a1d-6748051e7f34 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:25:34.578866 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-9be6b8ff-46a4-4a2a-8a1d-6748051e7f34 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:25:34.579778 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-9be6b8ff-46a4-4a2a-8a1d-6748051e7f34 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:25:34.580088 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-9be6b8ff-46a4-4a2a-8a1d-6748051e7f34 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:25:34.580519 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-9be6b8ff-46a4-4a2a-8a1d-6748051e7f34 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:25:34.619005 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-5892c397-a1a5-4f4d-93c1-787ca4e884a9 tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Calling method '>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:25:34.619622 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-5892c397-a1a5-4f4d-93c1-787ca4e884a9 tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] [instance: e49199b7-e998-44fb-9995-e6a13fd44dc6] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:25:34.625740 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-5892c397-a1a5-4f4d-93c1-787ca4e884a9 tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:25:34.626216 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-5892c397-a1a5-4f4d-93c1-787ca4e884a9 tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:25:34.626640 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-5892c397-a1a5-4f4d-93c1-787ca4e884a9 tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:25:34.627323 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-5892c397-a1a5-4f4d-93c1-787ca4e884a9 tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:25:34.627763 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-5892c397-a1a5-4f4d-93c1-787ca4e884a9 tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:25:34.628173 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-5892c397-a1a5-4f4d-93c1-787ca4e884a9 tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:25:34.658948 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-5892c397-a1a5-4f4d-93c1-787ca4e884a9 tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] [instance: e49199b7-e998-44fb-9995-e6a13fd44dc6] Going to try to terminate instance {{(pid=120864) delete /opt/stack/nova/nova/compute/api.py:2719}} Aug 27 02:25:34.726211 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG dbcounter [-] [120864] Writing DB stats nova_cell1:SELECT=99,nova_cell1:UPDATE=1 {{(pid=120864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 27 02:25:34.729329 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-9be6b8ff-46a4-4a2a-8a1d-6748051e7f34 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd945d512ae034ec08838d0a3afde2429', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '880ac4003248441c809a76cfd1b221eb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:25:34.738904 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-9be6b8ff-46a4-4a2a-8a1d-6748051e7f34 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] 10.222.0.52 "GET /compute/v2.1/servers/e2bf7024-0779-4e08-844a-43b78187e770" status: 200 len: 1502 microversion: 2.1 time: 0.175708 Aug 27 02:25:34.739528 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 200/399] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:25:34 2022] GET /compute/v2.1/servers/e2bf7024-0779-4e08-844a-43b78187e770 => generated 1502 bytes in 177 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:25:34.759498 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-bd12836c-82c8-4922-b4ca-c4e349095533 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "e6cc5290-e382-460d-a009-9a1c4c3ec685"}} {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 27 02:25:34.762136 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-bd12836c-82c8-4922-b4ca-c4e349095533 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] [instance: e2bf7024-0779-4e08-844a-43b78187e770] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:25:34.767370 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-5892c397-a1a5-4f4d-93c1-787ca4e884a9 tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] 10.222.0.52 "DELETE /compute/v2.1/servers/e49199b7-e998-44fb-9995-e6a13fd44dc6" status: 204 len: 0 microversion: 2.1 time: 0.151904 Aug 27 02:25:34.767935 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 200/400] 10.222.0.52 () {58 vars in 1239 bytes} [Sat Aug 27 02:25:34 2022] DELETE /compute/v2.1/servers/e49199b7-e998-44fb-9995-e6a13fd44dc6 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Aug 27 02:25:34.771625 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-bd12836c-82c8-4922-b4ca-c4e349095533 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:25:34.772082 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-bd12836c-82c8-4922-b4ca-c4e349095533 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:25:34.772565 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-bd12836c-82c8-4922-b4ca-c4e349095533 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:25:34.773263 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-bd12836c-82c8-4922-b4ca-c4e349095533 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:25:34.773712 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-bd12836c-82c8-4922-b4ca-c4e349095533 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:25:34.774097 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-bd12836c-82c8-4922-b4ca-c4e349095533 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:25:34.778003 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG dbcounter [-] [120863] Writing DB stats nova_cell1:SELECT=100 {{(pid=120863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 27 02:25:34.784943 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-5955a9f9-10e7-4767-82bc-cdf8c83e3bc7 tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Calling method '>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:25:34.786277 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-5955a9f9-10e7-4767-82bc-cdf8c83e3bc7 tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] [instance: e49199b7-e998-44fb-9995-e6a13fd44dc6] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:25:34.792838 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-5955a9f9-10e7-4767-82bc-cdf8c83e3bc7 tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:25:34.793425 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-5955a9f9-10e7-4767-82bc-cdf8c83e3bc7 tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:25:34.794142 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-5955a9f9-10e7-4767-82bc-cdf8c83e3bc7 tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:25:34.795082 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-5955a9f9-10e7-4767-82bc-cdf8c83e3bc7 tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:25:34.795699 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-5955a9f9-10e7-4767-82bc-cdf8c83e3bc7 tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:25:34.796266 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-5955a9f9-10e7-4767-82bc-cdf8c83e3bc7 tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:25:34.802091 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.wsgi [None req-bd12836c-82c8-4922-b4ca-c4e349095533 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] HTTP exception thrown: Cannot 'attach_volume' instance e2bf7024-0779-4e08-844a-43b78187e770 while it is in vm_state rescued Aug 27 02:25:34.804181 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-bd12836c-82c8-4922-b4ca-c4e349095533 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Request method failure captured: Aug 27 02:25:34.804181 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: request: POST /compute/v2.1/servers/e2bf7024-0779-4e08-844a-43b78187e770/os-volume_attachments HTTP/1.1 Aug 27 02:25:34.804181 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: Accept: application/json Aug 27 02:25:34.804181 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: Accept-Encoding: identity Aug 27 02:25:34.804181 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: Connection: close Aug 27 02:25:34.804181 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: Content-Length: 74 Aug 27 02:25:34.804181 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: Content-Type: application/json Aug 27 02:25:34.804181 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: Host: 10.222.0.52 Aug 27 02:25:34.804181 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: Openstack-System-Scope: None Aug 27 02:25:34.804181 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: User-Agent: python-urllib3/1.26.11 Aug 27 02:25:34.804181 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-Auth-Token: gAAAAABjCX_mTgf5582eaPxg7VeVEFNV70u35gVl4tYbLlqoLLSOTdT51sTqNUwxXgJqsRZlTyjbNrsB5eN_l1XXJng0l2G5rQDKOgGEffO457OUUMgEUK1evfhiyjkwW8ZvB7sHsdxG4zKRGU2G452QU2-KddZbqwFS3zX2uIy2nVW08GLJmCc Aug 27 02:25:34.804181 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-Domain-Id: None Aug 27 02:25:34.804181 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-Domain-Name: None Aug 27 02:25:34.804181 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-Identity-Status: Confirmed Aug 27 02:25:34.804181 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-Is-Admin-Project: True Aug 27 02:25:34.804181 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-Project-Domain-Id: default Aug 27 02:25:34.804181 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-Project-Domain-Name: Default Aug 27 02:25:34.804181 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-Project-Id: 880ac4003248441c809a76cfd1b221eb Aug 27 02:25:34.804181 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-Project-Name: tempest-ServerRescueNegativeTestJSON-1551042004 Aug 27 02:25:34.804181 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-Role: reader,member Aug 27 02:25:34.804181 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-Roles: reader,member Aug 27 02:25:34.805469 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-Service-Catalog: [{"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.52:8080", "publicURL": "http://10.222.0.52:8080/v1/AUTH_880ac4003248441c809a76cfd1b221eb"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52:9696/networking"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/compute/v2/880ac4003248441c809a76cfd1b221eb"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/image"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/volume/v3/880ac4003248441c809a76cfd1b221eb"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/identity"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/volume/v3/880ac4003248441c809a76cfd1b221eb"}]}] Aug 27 02:25:34.805469 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-Tenant: tempest-ServerRescueNegativeTestJSON-1551042004 Aug 27 02:25:34.805469 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-Tenant-Id: 880ac4003248441c809a76cfd1b221eb Aug 27 02:25:34.805469 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-Tenant-Name: tempest-ServerRescueNegativeTestJSON-1551042004 Aug 27 02:25:34.805469 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-User: tempest-ServerRescueNegativeTestJSON-1551042004-project Aug 27 02:25:34.805469 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-User-Domain-Id: default Aug 27 02:25:34.805469 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-User-Domain-Name: Default Aug 27 02:25:34.805469 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-User-Id: d945d512ae034ec08838d0a3afde2429 Aug 27 02:25:34.805469 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-User-Name: tempest-ServerRescueNegativeTestJSON-1551042004-project Aug 27 02:25:34.805469 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: Aug 27 02:25:34.805469 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: {"volumeAttachment": {"volumeId": "e6cc5290-e382-460d-a009-9a1c4c3ec685"}} Aug 27 02:25:34.805469 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: method: > Aug 27 02:25:34.806756 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: exception: Cannot 'attach_volume' instance e2bf7024-0779-4e08-844a-43b78187e770 while it is in vm_state rescued Aug 27 02:25:34.806756 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: action_args: {'server_id': 'e2bf7024-0779-4e08-844a-43b78187e770', 'body': {'volumeAttachment': {'volumeId': 'e6cc5290-e382-460d-a009-9a1c4c3ec685'}}} Aug 27 02:25:34.806756 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Aug 27 02:25:34.806756 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 27 02:25:34.806756 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 388, in create Aug 27 02:25:34.806756 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi device = self.compute_api.attach_volume( Aug 27 02:25:34.806756 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 241, in inner Aug 27 02:25:34.806756 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Aug 27 02:25:34.806756 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 148, in inner Aug 27 02:25:34.806756 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi raise exception.InstanceInvalidState( Aug 27 02:25:34.806756 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi nova.exception.InstanceInvalidState: Instance e2bf7024-0779-4e08-844a-43b78187e770 in vm_state rescued. Cannot attach_volume while the instance is in this state. Aug 27 02:25:34.806756 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi Aug 27 02:25:34.806756 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 27 02:25:34.806756 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi Aug 27 02:25:34.806756 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 27 02:25:34.806756 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 27 02:25:34.806756 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 27 02:25:34.806756 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Aug 27 02:25:34.806756 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Aug 27 02:25:34.812441 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Aug 27 02:25:34.812441 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Aug 27 02:25:34.812441 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Aug 27 02:25:34.812441 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Aug 27 02:25:34.812441 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Aug 27 02:25:34.812441 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Aug 27 02:25:34.812441 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Aug 27 02:25:34.812441 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Aug 27 02:25:34.812441 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 398, in create Aug 27 02:25:34.812441 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi common.raise_http_conflict_for_instance_invalid_state(state_error, Aug 27 02:25:34.812441 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 347, in raise_http_conflict_for_instance_invalid_state Aug 27 02:25:34.812441 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPConflict(explanation=msg) Aug 27 02:25:34.812441 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi webob.exc.HTTPConflict: Cannot 'attach_volume' instance e2bf7024-0779-4e08-844a-43b78187e770 while it is in vm_state rescued Aug 27 02:25:34.812441 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi Aug 27 02:25:34.812441 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 27 02:25:34.812441 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi Aug 27 02:25:34.812441 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 27 02:25:34.812441 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 27 02:25:34.812441 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 27 02:25:34.815234 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Aug 27 02:25:34.815234 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Aug 27 02:25:34.815234 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Cannot 'attach_volume' instance e2bf7024-0779-4e08-844a-43b78187e770 while it is in vm_state rescued Aug 27 02:25:34.815234 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi Aug 27 02:25:34.815234 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-bd12836c-82c8-4922-b4ca-c4e349095533 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Returning 409 to user: Cannot 'attach_volume' instance e2bf7024-0779-4e08-844a-43b78187e770 while it is in vm_state rescued {{(pid=120863) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Aug 27 02:25:34.815234 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-bd12836c-82c8-4922-b4ca-c4e349095533 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] 10.222.0.52 "POST /compute/v2.1/servers/e2bf7024-0779-4e08-844a-43b78187e770/os-volume_attachments" status: 409 len: 152 microversion: 2.1 time: 0.051550 Aug 27 02:25:34.815234 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 201/401] 10.222.0.52 () {60 vars in 1323 bytes} [Sat Aug 27 02:25:34 2022] POST /compute/v2.1/servers/e2bf7024-0779-4e08-844a-43b78187e770/os-volume_attachments => generated 152 bytes in 53 msecs (HTTP/1.1 409) 9 headers in 378 bytes (2 switches on core 0) Aug 27 02:25:34.820021 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-3d65bf44-6c47-4e1c-81f2-c237b3fa1d26 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Action: 'action', calling method: >, body: {"unrescue": {}} {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 27 02:25:34.820638 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-3d65bf44-6c47-4e1c-81f2-c237b3fa1d26 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] [instance: e2bf7024-0779-4e08-844a-43b78187e770] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:25:34.832363 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-3d65bf44-6c47-4e1c-81f2-c237b3fa1d26 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:25:34.834314 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-3d65bf44-6c47-4e1c-81f2-c237b3fa1d26 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:25:34.834314 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-3d65bf44-6c47-4e1c-81f2-c237b3fa1d26 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:25:34.835052 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-3d65bf44-6c47-4e1c-81f2-c237b3fa1d26 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:25:34.835514 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-3d65bf44-6c47-4e1c-81f2-c237b3fa1d26 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:25:34.835967 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-3d65bf44-6c47-4e1c-81f2-c237b3fa1d26 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:25:34.908841 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-3d65bf44-6c47-4e1c-81f2-c237b3fa1d26 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] 10.222.0.52 "POST /compute/v2.1/servers/e2bf7024-0779-4e08-844a-43b78187e770/action" status: 202 len: 0 microversion: 2.1 time: 0.095722 Aug 27 02:25:34.912742 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 202/402] 10.222.0.52 () {60 vars in 1278 bytes} [Sat Aug 27 02:25:34 2022] POST /compute/v2.1/servers/e2bf7024-0779-4e08-844a-43b78187e770/action => generated 0 bytes in 97 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Aug 27 02:25:34.918541 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-3fc7caaa-80e3-4238-a051-45cb96481af9 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:25:34.919326 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-3fc7caaa-80e3-4238-a051-45cb96481af9 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] [instance: e2bf7024-0779-4e08-844a-43b78187e770] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:25:34.926706 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-3fc7caaa-80e3-4238-a051-45cb96481af9 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:25:34.927272 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-3fc7caaa-80e3-4238-a051-45cb96481af9 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:25:34.927875 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-3fc7caaa-80e3-4238-a051-45cb96481af9 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:25:34.928663 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-3fc7caaa-80e3-4238-a051-45cb96481af9 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:25:34.929237 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-3fc7caaa-80e3-4238-a051-45cb96481af9 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:25:34.929663 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-3fc7caaa-80e3-4238-a051-45cb96481af9 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:25:35.056345 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-5955a9f9-10e7-4767-82bc-cdf8c83e3bc7 tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2e60cf2931f04f0e8e911adea32979dd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7cb3e3496c1a45c8942c3678c0c299e5', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:25:35.064344 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-5955a9f9-10e7-4767-82bc-cdf8c83e3bc7 tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] 10.222.0.52 "GET /compute/v2.1/servers/e49199b7-e998-44fb-9995-e6a13fd44dc6" status: 200 len: 1501 microversion: 2.1 time: 0.285829 Aug 27 02:25:35.065531 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 201/403] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:25:34 2022] GET /compute/v2.1/servers/e49199b7-e998-44fb-9995-e6a13fd44dc6 => generated 1501 bytes in 288 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:25:35.070537 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-d1af882c-2aec-482a-b86e-ab23c50c7eeb tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-TestShelveInstance-server-1737433225", "imageRef": "", "flavorRef": "452", "key_name": "tempest-TestShelveInstance-1152661178", "security_groups": [{"name": "tempest-secgroup-smoke-580183686"}], "networks": [{"uuid": "cc2c2e5a-cfe4-4fb7-a6ca-02f8ceda0efd"}], "block_device_mapping_v2": [{"uuid": "6bb4602b-7a42-427d-977e-1ef61460765d", "source_type": "volume", "destination_type": "volume", "boot_index": 0, "delete_on_termination": true}]}} {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 27 02:25:35.116269 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-3fc7caaa-80e3-4238-a051-45cb96481af9 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd945d512ae034ec08838d0a3afde2429', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '880ac4003248441c809a76cfd1b221eb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:25:35.124096 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-3fc7caaa-80e3-4238-a051-45cb96481af9 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] 10.222.0.52 "GET /compute/v2.1/servers/e2bf7024-0779-4e08-844a-43b78187e770" status: 200 len: 1510 microversion: 2.1 time: 0.209628 Aug 27 02:25:35.124667 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 203/404] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:25:34 2022] GET /compute/v2.1/servers/e2bf7024-0779-4e08-844a-43b78187e770 => generated 1510 bytes in 211 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 27 02:25:35.198907 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.network.neutron [None req-d1af882c-2aec-482a-b86e-ab23c50c7eeb tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] validate_networks() for [('cc2c2e5a-cfe4-4fb7-a6ca-02f8ceda0efd', None, None, None, None, None)] {{(pid=120864) validate_networks /opt/stack/nova/nova/network/neutron.py:2626}} Aug 27 02:25:35.600318 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.virt.hardware [None req-d1af882c-2aec-482a-b86e-ab23c50c7eeb tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] Flavor limits 0:0:0 {{(pid=120864) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Aug 27 02:25:35.600769 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.virt.hardware [None req-d1af882c-2aec-482a-b86e-ab23c50c7eeb tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] Image limits 0:0:0 {{(pid=120864) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Aug 27 02:25:35.601269 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.virt.hardware [None req-d1af882c-2aec-482a-b86e-ab23c50c7eeb tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] Flavor pref 0:0:0 {{(pid=120864) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Aug 27 02:25:35.601703 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.virt.hardware [None req-d1af882c-2aec-482a-b86e-ab23c50c7eeb tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] Image pref 0:0:0 {{(pid=120864) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Aug 27 02:25:35.602110 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.virt.hardware [None req-d1af882c-2aec-482a-b86e-ab23c50c7eeb tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=120864) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Aug 27 02:25:35.611158 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-d1af882c-2aec-482a-b86e-ab23c50c7eeb tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:25:35.611627 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-d1af882c-2aec-482a-b86e-ab23c50c7eeb tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:25:35.612066 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-d1af882c-2aec-482a-b86e-ab23c50c7eeb tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:25:35.612805 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-d1af882c-2aec-482a-b86e-ab23c50c7eeb tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:25:35.613450 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-d1af882c-2aec-482a-b86e-ab23c50c7eeb tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:25:35.613889 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-d1af882c-2aec-482a-b86e-ab23c50c7eeb tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:25:35.635069 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.quota [None req-d1af882c-2aec-482a-b86e-ab23c50c7eeb tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] Getting quotas for project b2ddff853b7546788b645a0344c2297b. Resources: {'cores', 'ram', 'instances'} {{(pid=120864) _get_quotas /opt/stack/nova/nova/quota.py:393}} Aug 27 02:25:35.639794 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.quota [None req-d1af882c-2aec-482a-b86e-ab23c50c7eeb tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] Getting quotas for user 642291c4daa04a6ab3fbce92a8be39c6 and project b2ddff853b7546788b645a0344c2297b. Resources: {'cores', 'ram', 'instances'} {{(pid=120864) _get_quotas /opt/stack/nova/nova/quota.py:383}} Aug 27 02:25:35.699370 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-d1af882c-2aec-482a-b86e-ab23c50c7eeb tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] Going to run 1 instances... {{(pid=120864) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Aug 27 02:25:35.701945 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-d1af882c-2aec-482a-b86e-ab23c50c7eeb tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] [instance: 9941c49f-8626-461d-a570-c0ffbfd4b804] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='volume',device_name=None,device_type=None,disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id=None,instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='volume',tag=None,updated_at=,uuid=,volume_id='6bb4602b-7a42-427d-977e-1ef61460765d',volume_size=None,volume_type=None)] {{(pid=120864) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Aug 27 02:25:35.858303 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-d1af882c-2aec-482a-b86e-ab23c50c7eeb tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] 10.222.0.52 "POST /compute/v2.1/servers" status: 202 len: 405 microversion: 2.1 time: 0.791376 Aug 27 02:25:35.858941 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 202/405] 10.222.0.52 () {60 vars in 1147 bytes} [Sat Aug 27 02:25:35 2022] POST /compute/v2.1/servers => generated 405 bytes in 793 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Aug 27 02:25:35.870265 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-6c65e97b-b87d-458b-b14b-db78cc258ed5 tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:25:35.870872 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-6c65e97b-b87d-458b-b14b-db78cc258ed5 tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] [instance: 9941c49f-8626-461d-a570-c0ffbfd4b804] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:25:35.926262 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-6c65e97b-b87d-458b-b14b-db78cc258ed5 tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '642291c4daa04a6ab3fbce92a8be39c6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b2ddff853b7546788b645a0344c2297b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:25:35.933364 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-6c65e97b-b87d-458b-b14b-db78cc258ed5 tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] 10.222.0.52 "GET /compute/v2.1/servers/9941c49f-8626-461d-a570-c0ffbfd4b804" status: 200 len: 1089 microversion: 2.1 time: 0.066902 Aug 27 02:25:35.940071 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 204/406] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:25:35 2022] GET /compute/v2.1/servers/9941c49f-8626-461d-a570-c0ffbfd4b804 => generated 1089 bytes in 68 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:25:36.088225 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-95ae8ef4-3cb4-4992-b9f1-8b3d71adff97 tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Calling method '>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:25:36.089476 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-95ae8ef4-3cb4-4992-b9f1-8b3d71adff97 tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] [instance: e49199b7-e998-44fb-9995-e6a13fd44dc6] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:25:36.096042 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-95ae8ef4-3cb4-4992-b9f1-8b3d71adff97 tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:25:36.096597 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-95ae8ef4-3cb4-4992-b9f1-8b3d71adff97 tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:25:36.097094 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-95ae8ef4-3cb4-4992-b9f1-8b3d71adff97 tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:25:36.098516 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-95ae8ef4-3cb4-4992-b9f1-8b3d71adff97 tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:25:36.099046 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-95ae8ef4-3cb4-4992-b9f1-8b3d71adff97 tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:25:36.099530 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-95ae8ef4-3cb4-4992-b9f1-8b3d71adff97 tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:25:36.150371 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-12244ceb-e752-48cc-bb2c-06f505e52b7c tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:25:36.150985 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-12244ceb-e752-48cc-bb2c-06f505e52b7c tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] [instance: e2bf7024-0779-4e08-844a-43b78187e770] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:25:36.168793 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-12244ceb-e752-48cc-bb2c-06f505e52b7c tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:25:36.169329 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-12244ceb-e752-48cc-bb2c-06f505e52b7c tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:25:36.169846 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-12244ceb-e752-48cc-bb2c-06f505e52b7c tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:25:36.170654 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-12244ceb-e752-48cc-bb2c-06f505e52b7c tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:25:36.171613 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-12244ceb-e752-48cc-bb2c-06f505e52b7c tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:25:36.172400 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-12244ceb-e752-48cc-bb2c-06f505e52b7c tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:25:36.427724 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-95ae8ef4-3cb4-4992-b9f1-8b3d71adff97 tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2e60cf2931f04f0e8e911adea32979dd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7cb3e3496c1a45c8942c3678c0c299e5', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:25:36.436369 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-95ae8ef4-3cb4-4992-b9f1-8b3d71adff97 tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] 10.222.0.52 "GET /compute/v2.1/servers/e49199b7-e998-44fb-9995-e6a13fd44dc6" status: 200 len: 1501 microversion: 2.1 time: 0.351982 Aug 27 02:25:36.446202 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 203/407] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:25:36 2022] GET /compute/v2.1/servers/e49199b7-e998-44fb-9995-e6a13fd44dc6 => generated 1501 bytes in 362 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:25:36.567388 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-12244ceb-e752-48cc-bb2c-06f505e52b7c tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd945d512ae034ec08838d0a3afde2429', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '880ac4003248441c809a76cfd1b221eb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:25:36.577684 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-12244ceb-e752-48cc-bb2c-06f505e52b7c tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] 10.222.0.52 "GET /compute/v2.1/servers/e2bf7024-0779-4e08-844a-43b78187e770" status: 200 len: 1510 microversion: 2.1 time: 0.435283 Aug 27 02:25:36.592442 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 205/408] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:25:36 2022] GET /compute/v2.1/servers/e2bf7024-0779-4e08-844a-43b78187e770 => generated 1510 bytes in 451 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:25:36.958660 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-56765e55-1f0d-43f5-8a33-aa93a7b3305f tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] Calling method '>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:25:36.960304 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-56765e55-1f0d-43f5-8a33-aa93a7b3305f tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] [instance: 9941c49f-8626-461d-a570-c0ffbfd4b804] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:25:36.966814 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-56765e55-1f0d-43f5-8a33-aa93a7b3305f tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:25:36.967521 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-56765e55-1f0d-43f5-8a33-aa93a7b3305f tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:25:36.968020 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-56765e55-1f0d-43f5-8a33-aa93a7b3305f tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:25:36.968896 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-56765e55-1f0d-43f5-8a33-aa93a7b3305f tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:25:36.970341 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-56765e55-1f0d-43f5-8a33-aa93a7b3305f tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:25:36.970887 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-56765e55-1f0d-43f5-8a33-aa93a7b3305f tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:25:37.162677 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-56765e55-1f0d-43f5-8a33-aa93a7b3305f tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '642291c4daa04a6ab3fbce92a8be39c6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b2ddff853b7546788b645a0344c2297b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:25:37.178204 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-56765e55-1f0d-43f5-8a33-aa93a7b3305f tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] 10.222.0.52 "GET /compute/v2.1/servers/9941c49f-8626-461d-a570-c0ffbfd4b804" status: 200 len: 1131 microversion: 2.1 time: 0.227007 Aug 27 02:25:37.195199 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 204/409] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:25:36 2022] GET /compute/v2.1/servers/9941c49f-8626-461d-a570-c0ffbfd4b804 => generated 1131 bytes in 245 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 27 02:25:37.468234 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-8eada2d1-81fd-4141-a432-80dab7046b8b tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:25:37.469725 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-8eada2d1-81fd-4141-a432-80dab7046b8b tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] [instance: e49199b7-e998-44fb-9995-e6a13fd44dc6] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:25:37.478072 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-8eada2d1-81fd-4141-a432-80dab7046b8b tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:25:37.478954 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-8eada2d1-81fd-4141-a432-80dab7046b8b tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:25:37.479968 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-8eada2d1-81fd-4141-a432-80dab7046b8b tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:25:37.480998 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-8eada2d1-81fd-4141-a432-80dab7046b8b tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:25:37.482048 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-8eada2d1-81fd-4141-a432-80dab7046b8b tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:25:37.482626 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-8eada2d1-81fd-4141-a432-80dab7046b8b tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:25:37.599721 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-6e0725a7-7dbb-43e9-8086-f6f8a999bdfb tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Calling method '>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:25:37.600320 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-6e0725a7-7dbb-43e9-8086-f6f8a999bdfb tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] [instance: e2bf7024-0779-4e08-844a-43b78187e770] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:25:37.607434 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-6e0725a7-7dbb-43e9-8086-f6f8a999bdfb tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:25:37.608374 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-6e0725a7-7dbb-43e9-8086-f6f8a999bdfb tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:25:37.609050 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-6e0725a7-7dbb-43e9-8086-f6f8a999bdfb tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:25:37.610076 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-6e0725a7-7dbb-43e9-8086-f6f8a999bdfb tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:25:37.610668 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-6e0725a7-7dbb-43e9-8086-f6f8a999bdfb tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:25:37.611229 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-6e0725a7-7dbb-43e9-8086-f6f8a999bdfb tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:25:37.673978 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-8eada2d1-81fd-4141-a432-80dab7046b8b tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2e60cf2931f04f0e8e911adea32979dd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7cb3e3496c1a45c8942c3678c0c299e5', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:25:37.682213 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-8eada2d1-81fd-4141-a432-80dab7046b8b tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] 10.222.0.52 "GET /compute/v2.1/servers/e49199b7-e998-44fb-9995-e6a13fd44dc6" status: 200 len: 1501 microversion: 2.1 time: 0.217983 Aug 27 02:25:37.682906 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 206/410] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:25:37 2022] GET /compute/v2.1/servers/e49199b7-e998-44fb-9995-e6a13fd44dc6 => generated 1501 bytes in 220 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 27 02:25:37.790444 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-6e0725a7-7dbb-43e9-8086-f6f8a999bdfb tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd945d512ae034ec08838d0a3afde2429', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '880ac4003248441c809a76cfd1b221eb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:25:37.799532 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-6e0725a7-7dbb-43e9-8086-f6f8a999bdfb tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] 10.222.0.52 "GET /compute/v2.1/servers/e2bf7024-0779-4e08-844a-43b78187e770" status: 200 len: 1510 microversion: 2.1 time: 0.203787 Aug 27 02:25:37.800539 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 205/411] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:25:37 2022] GET /compute/v2.1/servers/e2bf7024-0779-4e08-844a-43b78187e770 => generated 1510 bytes in 206 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:25:38.058516 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [req-3b608e53-6639-446c-94f8-8792d192c220 req-ccd93553-9b0e-45a2-bb2c-53237881d743 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "e49199b7-e998-44fb-9995-e6a13fd44dc6", "name": "network-vif-deleted", "tag": "55122ed4-7c38-4c0f-b4b9-a09312935cbc"}]} {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 27 02:25:38.070860 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [req-3b608e53-6639-446c-94f8-8792d192c220 req-ccd93553-9b0e-45a2-bb2c-53237881d743 service nova] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:25:38.071341 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [req-3b608e53-6639-446c-94f8-8792d192c220 req-ccd93553-9b0e-45a2-bb2c-53237881d743 service nova] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:25:38.071770 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [req-3b608e53-6639-446c-94f8-8792d192c220 req-ccd93553-9b0e-45a2-bb2c-53237881d743 service nova] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:25:38.084232 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.compute.server_external_events [req-3b608e53-6639-446c-94f8-8792d192c220 req-ccd93553-9b0e-45a2-bb2c-53237881d743 service nova] Creating event network-vif-deleted:55122ed4-7c38-4c0f-b4b9-a09312935cbc for instance e49199b7-e998-44fb-9995-e6a13fd44dc6 on hv-cinder-84630 Aug 27 02:25:38.089183 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [req-3b608e53-6639-446c-94f8-8792d192c220 req-ccd93553-9b0e-45a2-bb2c-53237881d743 service nova] 10.222.0.52 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.034582 Aug 27 02:25:38.089804 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 207/412] 10.222.0.52 () {64 vars in 1317 bytes} [Sat Aug 27 02:25:38 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 36 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 27 02:25:38.205733 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-633bcd6c-5564-4a6f-a549-37da87d2e9da tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] Calling method '>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:25:38.206848 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-633bcd6c-5564-4a6f-a549-37da87d2e9da tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] [instance: 9941c49f-8626-461d-a570-c0ffbfd4b804] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:25:38.216310 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-633bcd6c-5564-4a6f-a549-37da87d2e9da tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:25:38.216895 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-633bcd6c-5564-4a6f-a549-37da87d2e9da tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:25:38.217528 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-633bcd6c-5564-4a6f-a549-37da87d2e9da tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:25:38.219042 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-633bcd6c-5564-4a6f-a549-37da87d2e9da tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:25:38.219813 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-633bcd6c-5564-4a6f-a549-37da87d2e9da tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:25:38.220682 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-633bcd6c-5564-4a6f-a549-37da87d2e9da tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:25:38.290649 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-633bcd6c-5564-4a6f-a549-37da87d2e9da tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '642291c4daa04a6ab3fbce92a8be39c6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b2ddff853b7546788b645a0344c2297b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:25:38.298987 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-633bcd6c-5564-4a6f-a549-37da87d2e9da tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] 10.222.0.52 "GET /compute/v2.1/servers/9941c49f-8626-461d-a570-c0ffbfd4b804" status: 200 len: 1187 microversion: 2.1 time: 0.099246 Aug 27 02:25:38.299870 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 206/413] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:25:38 2022] GET /compute/v2.1/servers/9941c49f-8626-461d-a570-c0ffbfd4b804 => generated 1187 bytes in 101 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:25:38.705200 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-2c5759e8-f2ea-4bec-8ec6-a84a0014aa9b tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:25:38.705793 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-2c5759e8-f2ea-4bec-8ec6-a84a0014aa9b tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] [instance: e49199b7-e998-44fb-9995-e6a13fd44dc6] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:25:38.713252 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-2c5759e8-f2ea-4bec-8ec6-a84a0014aa9b tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:25:38.714000 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-2c5759e8-f2ea-4bec-8ec6-a84a0014aa9b tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:25:38.714626 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-2c5759e8-f2ea-4bec-8ec6-a84a0014aa9b tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:25:38.715563 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-2c5759e8-f2ea-4bec-8ec6-a84a0014aa9b tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:25:38.716087 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-2c5759e8-f2ea-4bec-8ec6-a84a0014aa9b tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:25:38.716745 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-2c5759e8-f2ea-4bec-8ec6-a84a0014aa9b tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:25:38.788982 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-2c5759e8-f2ea-4bec-8ec6-a84a0014aa9b tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2e60cf2931f04f0e8e911adea32979dd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7cb3e3496c1a45c8942c3678c0c299e5', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:25:38.796769 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-2c5759e8-f2ea-4bec-8ec6-a84a0014aa9b tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] 10.222.0.52 "GET /compute/v2.1/servers/e49199b7-e998-44fb-9995-e6a13fd44dc6" status: 200 len: 1459 microversion: 2.1 time: 0.096837 Aug 27 02:25:38.797443 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 208/414] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:25:38 2022] GET /compute/v2.1/servers/e49199b7-e998-44fb-9995-e6a13fd44dc6 => generated 1459 bytes in 98 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:25:38.824350 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-4bbcc56c-0038-4c8c-b679-63d2a7c4cc12 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Calling method '>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:25:38.824952 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-4bbcc56c-0038-4c8c-b679-63d2a7c4cc12 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] [instance: e2bf7024-0779-4e08-844a-43b78187e770] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:25:38.832535 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-4bbcc56c-0038-4c8c-b679-63d2a7c4cc12 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:25:38.833207 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-4bbcc56c-0038-4c8c-b679-63d2a7c4cc12 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:25:38.833633 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-4bbcc56c-0038-4c8c-b679-63d2a7c4cc12 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:25:38.834358 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-4bbcc56c-0038-4c8c-b679-63d2a7c4cc12 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:25:38.835111 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-4bbcc56c-0038-4c8c-b679-63d2a7c4cc12 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:25:38.835820 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-4bbcc56c-0038-4c8c-b679-63d2a7c4cc12 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:25:38.973404 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-4bbcc56c-0038-4c8c-b679-63d2a7c4cc12 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd945d512ae034ec08838d0a3afde2429', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '880ac4003248441c809a76cfd1b221eb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:25:38.981428 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-4bbcc56c-0038-4c8c-b679-63d2a7c4cc12 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] 10.222.0.52 "GET /compute/v2.1/servers/e2bf7024-0779-4e08-844a-43b78187e770" status: 200 len: 1510 microversion: 2.1 time: 0.161085 Aug 27 02:25:38.982031 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 207/415] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:25:38 2022] GET /compute/v2.1/servers/e2bf7024-0779-4e08-844a-43b78187e770 => generated 1510 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:25:39.320954 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-1bc771c7-879a-4f79-aa0b-82e99fd0cdcb tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:25:39.322015 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-1bc771c7-879a-4f79-aa0b-82e99fd0cdcb tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] [instance: 9941c49f-8626-461d-a570-c0ffbfd4b804] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:25:39.329167 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-1bc771c7-879a-4f79-aa0b-82e99fd0cdcb tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:25:39.329663 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-1bc771c7-879a-4f79-aa0b-82e99fd0cdcb tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:25:39.330084 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-1bc771c7-879a-4f79-aa0b-82e99fd0cdcb tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:25:39.330794 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-1bc771c7-879a-4f79-aa0b-82e99fd0cdcb tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:25:39.331235 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-1bc771c7-879a-4f79-aa0b-82e99fd0cdcb tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:25:39.331645 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-1bc771c7-879a-4f79-aa0b-82e99fd0cdcb tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:25:39.402599 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-1bc771c7-879a-4f79-aa0b-82e99fd0cdcb tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '642291c4daa04a6ab3fbce92a8be39c6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b2ddff853b7546788b645a0344c2297b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:25:39.410780 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-1bc771c7-879a-4f79-aa0b-82e99fd0cdcb tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] 10.222.0.52 "GET /compute/v2.1/servers/9941c49f-8626-461d-a570-c0ffbfd4b804" status: 200 len: 1187 microversion: 2.1 time: 0.094791 Aug 27 02:25:39.423991 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 209/416] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:25:39 2022] GET /compute/v2.1/servers/9941c49f-8626-461d-a570-c0ffbfd4b804 => generated 1187 bytes in 109 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:25:39.824554 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-9985404d-0b53-418c-984d-b87b08e2c69a tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Calling method '>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:25:39.825622 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-9985404d-0b53-418c-984d-b87b08e2c69a tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] [instance: e49199b7-e998-44fb-9995-e6a13fd44dc6] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:25:39.831762 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-9985404d-0b53-418c-984d-b87b08e2c69a tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:25:39.832325 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-9985404d-0b53-418c-984d-b87b08e2c69a tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:25:39.832997 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-9985404d-0b53-418c-984d-b87b08e2c69a tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:25:39.833722 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-9985404d-0b53-418c-984d-b87b08e2c69a tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:25:39.834218 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-9985404d-0b53-418c-984d-b87b08e2c69a tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:25:39.834803 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-9985404d-0b53-418c-984d-b87b08e2c69a tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:25:39.900222 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-9985404d-0b53-418c-984d-b87b08e2c69a tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2e60cf2931f04f0e8e911adea32979dd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7cb3e3496c1a45c8942c3678c0c299e5', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:25:39.908296 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-9985404d-0b53-418c-984d-b87b08e2c69a tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] 10.222.0.52 "GET /compute/v2.1/servers/e49199b7-e998-44fb-9995-e6a13fd44dc6" status: 200 len: 1298 microversion: 2.1 time: 0.088096 Aug 27 02:25:39.908880 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 208/417] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:25:39 2022] GET /compute/v2.1/servers/e49199b7-e998-44fb-9995-e6a13fd44dc6 => generated 1298 bytes in 90 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 27 02:25:40.002364 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-ed4efee1-ca78-494d-9655-b9b27f776751 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:25:40.002993 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-ed4efee1-ca78-494d-9655-b9b27f776751 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] [instance: e2bf7024-0779-4e08-844a-43b78187e770] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:25:40.020813 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-ed4efee1-ca78-494d-9655-b9b27f776751 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:25:40.021325 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-ed4efee1-ca78-494d-9655-b9b27f776751 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:25:40.021952 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-ed4efee1-ca78-494d-9655-b9b27f776751 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:25:40.022657 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-ed4efee1-ca78-494d-9655-b9b27f776751 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:25:40.023376 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-ed4efee1-ca78-494d-9655-b9b27f776751 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:25:40.023989 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-ed4efee1-ca78-494d-9655-b9b27f776751 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:25:40.164618 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-ed4efee1-ca78-494d-9655-b9b27f776751 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd945d512ae034ec08838d0a3afde2429', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '880ac4003248441c809a76cfd1b221eb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:25:40.174935 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-ed4efee1-ca78-494d-9655-b9b27f776751 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] 10.222.0.52 "GET /compute/v2.1/servers/e2bf7024-0779-4e08-844a-43b78187e770" status: 200 len: 1510 microversion: 2.1 time: 0.176790 Aug 27 02:25:40.176600 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 210/418] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:25:39 2022] GET /compute/v2.1/servers/e2bf7024-0779-4e08-844a-43b78187e770 => generated 1510 bytes in 179 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:25:40.430826 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-6edea761-2791-4f8c-8298-7658eb1f1d00 tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] Calling method '>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:25:40.431426 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-6edea761-2791-4f8c-8298-7658eb1f1d00 tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] [instance: 9941c49f-8626-461d-a570-c0ffbfd4b804] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:25:40.437677 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-6edea761-2791-4f8c-8298-7658eb1f1d00 tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:25:40.438124 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-6edea761-2791-4f8c-8298-7658eb1f1d00 tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:25:40.438795 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-6edea761-2791-4f8c-8298-7658eb1f1d00 tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:25:40.439624 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-6edea761-2791-4f8c-8298-7658eb1f1d00 tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:25:40.440254 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-6edea761-2791-4f8c-8298-7658eb1f1d00 tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:25:40.440919 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-6edea761-2791-4f8c-8298-7658eb1f1d00 tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:25:40.518886 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-6edea761-2791-4f8c-8298-7658eb1f1d00 tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '642291c4daa04a6ab3fbce92a8be39c6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b2ddff853b7546788b645a0344c2297b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:25:40.527153 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-6edea761-2791-4f8c-8298-7658eb1f1d00 tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] 10.222.0.52 "GET /compute/v2.1/servers/9941c49f-8626-461d-a570-c0ffbfd4b804" status: 200 len: 1195 microversion: 2.1 time: 0.100103 Aug 27 02:25:40.528538 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 209/419] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:25:40 2022] GET /compute/v2.1/servers/9941c49f-8626-461d-a570-c0ffbfd4b804 => generated 1195 bytes in 102 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:25:40.929596 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-0d0a1ee7-45e2-4a1f-9ae3-b6eff3ac9aa1 tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:25:40.930279 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-0d0a1ee7-45e2-4a1f-9ae3-b6eff3ac9aa1 tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] [instance: e49199b7-e998-44fb-9995-e6a13fd44dc6] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:25:40.936869 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-0d0a1ee7-45e2-4a1f-9ae3-b6eff3ac9aa1 tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:25:40.937429 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-0d0a1ee7-45e2-4a1f-9ae3-b6eff3ac9aa1 tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:25:40.937945 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-0d0a1ee7-45e2-4a1f-9ae3-b6eff3ac9aa1 tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:25:40.938736 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-0d0a1ee7-45e2-4a1f-9ae3-b6eff3ac9aa1 tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:25:40.939352 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-0d0a1ee7-45e2-4a1f-9ae3-b6eff3ac9aa1 tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:25:40.939839 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-0d0a1ee7-45e2-4a1f-9ae3-b6eff3ac9aa1 tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:25:40.972426 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-0d0a1ee7-45e2-4a1f-9ae3-b6eff3ac9aa1 tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:25:40.972795 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-0d0a1ee7-45e2-4a1f-9ae3-b6eff3ac9aa1 tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:25:40.973335 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-0d0a1ee7-45e2-4a1f-9ae3-b6eff3ac9aa1 tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:25:40.973783 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.objects.instance [None req-0d0a1ee7-45e2-4a1f-9ae3-b6eff3ac9aa1 tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Lazy-loading 'fault' on Instance uuid e49199b7-e998-44fb-9995-e6a13fd44dc6 {{(pid=120863) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Aug 27 02:25:41.018208 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-0d0a1ee7-45e2-4a1f-9ae3-b6eff3ac9aa1 tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2e60cf2931f04f0e8e911adea32979dd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7cb3e3496c1a45c8942c3678c0c299e5', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:25:41.026546 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-0d0a1ee7-45e2-4a1f-9ae3-b6eff3ac9aa1 tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] 10.222.0.52 "GET /compute/v2.1/servers/e49199b7-e998-44fb-9995-e6a13fd44dc6" status: 200 len: 1303 microversion: 2.1 time: 0.101221 Aug 27 02:25:41.027109 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 211/420] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:25:40 2022] GET /compute/v2.1/servers/e49199b7-e998-44fb-9995-e6a13fd44dc6 => generated 1303 bytes in 102 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:25:41.155520 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-5aa6516d-5d57-4282-a62a-bafd02e9d363 tempest-VolumesSnapshotTestJSON-1750456804 tempest-VolumesSnapshotTestJSON-1750456804-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-VolumesSnapshotTestJSON-instance-2010670535", "imageRef": "9fc30b36-b827-46f9-892d-30e808bc2b8a", "flavorRef": "452", "networks": [{"uuid": "c0b61970-d693-42b1-b7a6-dd3cf1b79e9d"}]}} {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 27 02:25:41.196425 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-b0b072b5-11dd-4fb6-a92b-59a63a48ef11 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:25:41.197337 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-b0b072b5-11dd-4fb6-a92b-59a63a48ef11 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] [instance: e2bf7024-0779-4e08-844a-43b78187e770] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:25:41.204643 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-b0b072b5-11dd-4fb6-a92b-59a63a48ef11 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:25:41.205174 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-b0b072b5-11dd-4fb6-a92b-59a63a48ef11 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:25:41.205629 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-b0b072b5-11dd-4fb6-a92b-59a63a48ef11 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:25:41.206386 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-b0b072b5-11dd-4fb6-a92b-59a63a48ef11 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:25:41.206876 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-b0b072b5-11dd-4fb6-a92b-59a63a48ef11 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:25:41.207279 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-b0b072b5-11dd-4fb6-a92b-59a63a48ef11 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:25:41.225081 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.network.neutron [None req-5aa6516d-5d57-4282-a62a-bafd02e9d363 tempest-VolumesSnapshotTestJSON-1750456804 tempest-VolumesSnapshotTestJSON-1750456804-project] validate_networks() for [('c0b61970-d693-42b1-b7a6-dd3cf1b79e9d', None, None, None, None, None)] {{(pid=120864) validate_networks /opt/stack/nova/nova/network/neutron.py:2626}} Aug 27 02:25:41.460572 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-b0b072b5-11dd-4fb6-a92b-59a63a48ef11 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd945d512ae034ec08838d0a3afde2429', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '880ac4003248441c809a76cfd1b221eb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:25:41.469170 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-b0b072b5-11dd-4fb6-a92b-59a63a48ef11 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] 10.222.0.52 "GET /compute/v2.1/servers/e2bf7024-0779-4e08-844a-43b78187e770" status: 200 len: 1516 microversion: 2.1 time: 0.276743 Aug 27 02:25:41.470081 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 212/421] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:25:41 2022] GET /compute/v2.1/servers/e2bf7024-0779-4e08-844a-43b78187e770 => generated 1516 bytes in 279 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 27 02:25:41.552074 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-87535040-44be-40ec-b62a-e3da2bf3d09f tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:25:41.552688 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-87535040-44be-40ec-b62a-e3da2bf3d09f tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] [instance: 9941c49f-8626-461d-a570-c0ffbfd4b804] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:25:41.558990 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-87535040-44be-40ec-b62a-e3da2bf3d09f tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:25:41.559700 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-87535040-44be-40ec-b62a-e3da2bf3d09f tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:25:41.560178 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-87535040-44be-40ec-b62a-e3da2bf3d09f tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:25:41.560875 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-87535040-44be-40ec-b62a-e3da2bf3d09f tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:25:41.561500 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-87535040-44be-40ec-b62a-e3da2bf3d09f tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:25:41.562095 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-87535040-44be-40ec-b62a-e3da2bf3d09f tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:25:41.638100 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-87535040-44be-40ec-b62a-e3da2bf3d09f tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '642291c4daa04a6ab3fbce92a8be39c6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b2ddff853b7546788b645a0344c2297b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:25:41.654194 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-87535040-44be-40ec-b62a-e3da2bf3d09f tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] 10.222.0.52 "GET /compute/v2.1/servers/9941c49f-8626-461d-a570-c0ffbfd4b804" status: 200 len: 1205 microversion: 2.1 time: 0.107415 Aug 27 02:25:41.654816 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 213/422] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:25:41 2022] GET /compute/v2.1/servers/9941c49f-8626-461d-a570-c0ffbfd4b804 => generated 1205 bytes in 110 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 27 02:25:41.772128 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.virt.hardware [None req-5aa6516d-5d57-4282-a62a-bafd02e9d363 tempest-VolumesSnapshotTestJSON-1750456804 tempest-VolumesSnapshotTestJSON-1750456804-project] Flavor limits 0:0:0 {{(pid=120864) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Aug 27 02:25:41.772543 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.virt.hardware [None req-5aa6516d-5d57-4282-a62a-bafd02e9d363 tempest-VolumesSnapshotTestJSON-1750456804 tempest-VolumesSnapshotTestJSON-1750456804-project] Image limits 0:0:0 {{(pid=120864) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Aug 27 02:25:41.773014 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.virt.hardware [None req-5aa6516d-5d57-4282-a62a-bafd02e9d363 tempest-VolumesSnapshotTestJSON-1750456804 tempest-VolumesSnapshotTestJSON-1750456804-project] Flavor pref 0:0:0 {{(pid=120864) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Aug 27 02:25:41.781339 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.virt.hardware [None req-5aa6516d-5d57-4282-a62a-bafd02e9d363 tempest-VolumesSnapshotTestJSON-1750456804 tempest-VolumesSnapshotTestJSON-1750456804-project] Image pref 0:0:0 {{(pid=120864) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Aug 27 02:25:41.781617 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.virt.hardware [None req-5aa6516d-5d57-4282-a62a-bafd02e9d363 tempest-VolumesSnapshotTestJSON-1750456804 tempest-VolumesSnapshotTestJSON-1750456804-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=120864) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Aug 27 02:25:41.805993 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-5aa6516d-5d57-4282-a62a-bafd02e9d363 tempest-VolumesSnapshotTestJSON-1750456804 tempest-VolumesSnapshotTestJSON-1750456804-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:25:41.806634 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-5aa6516d-5d57-4282-a62a-bafd02e9d363 tempest-VolumesSnapshotTestJSON-1750456804 tempest-VolumesSnapshotTestJSON-1750456804-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.007s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:25:41.810567 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-5aa6516d-5d57-4282-a62a-bafd02e9d363 tempest-VolumesSnapshotTestJSON-1750456804 tempest-VolumesSnapshotTestJSON-1750456804-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:25:41.815034 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-5aa6516d-5d57-4282-a62a-bafd02e9d363 tempest-VolumesSnapshotTestJSON-1750456804 tempest-VolumesSnapshotTestJSON-1750456804-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:25:41.818325 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-5aa6516d-5d57-4282-a62a-bafd02e9d363 tempest-VolumesSnapshotTestJSON-1750456804 tempest-VolumesSnapshotTestJSON-1750456804-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:25:41.818943 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-5aa6516d-5d57-4282-a62a-bafd02e9d363 tempest-VolumesSnapshotTestJSON-1750456804 tempest-VolumesSnapshotTestJSON-1750456804-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:25:41.869850 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.quota [None req-5aa6516d-5d57-4282-a62a-bafd02e9d363 tempest-VolumesSnapshotTestJSON-1750456804 tempest-VolumesSnapshotTestJSON-1750456804-project] Getting quotas for project fa43d54225bf44b283f9db5ddbbb13ca. Resources: {'cores', 'ram', 'instances'} {{(pid=120864) _get_quotas /opt/stack/nova/nova/quota.py:393}} Aug 27 02:25:41.874786 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.quota [None req-5aa6516d-5d57-4282-a62a-bafd02e9d363 tempest-VolumesSnapshotTestJSON-1750456804 tempest-VolumesSnapshotTestJSON-1750456804-project] Getting quotas for user 75744eccc3d24d4c9282cbe3a3314489 and project fa43d54225bf44b283f9db5ddbbb13ca. Resources: {'cores', 'ram', 'instances'} {{(pid=120864) _get_quotas /opt/stack/nova/nova/quota.py:383}} Aug 27 02:25:41.891693 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-5aa6516d-5d57-4282-a62a-bafd02e9d363 tempest-VolumesSnapshotTestJSON-1750456804 tempest-VolumesSnapshotTestJSON-1750456804-project] Going to run 1 instances... {{(pid=120864) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Aug 27 02:25:41.894457 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-5aa6516d-5d57-4282-a62a-bafd02e9d363 tempest-VolumesSnapshotTestJSON-1750456804 tempest-VolumesSnapshotTestJSON-1750456804-project] [instance: c31c26a8-fe5a-40fb-91a1-7b5251fd0daa] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='9fc30b36-b827-46f9-892d-30e808bc2b8a',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=120864) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Aug 27 02:25:41.963943 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-5aa6516d-5d57-4282-a62a-bafd02e9d363 tempest-VolumesSnapshotTestJSON-1750456804 tempest-VolumesSnapshotTestJSON-1750456804-project] 10.222.0.52 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 0.813138 Aug 27 02:25:41.965468 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 210/423] 10.222.0.52 () {60 vars in 1147 bytes} [Sat Aug 27 02:25:41 2022] POST /compute/v2.1/servers => generated 380 bytes in 816 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Aug 27 02:25:41.979162 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-f8c0b76d-6a1b-4705-b88c-902a59dc2bfa tempest-VolumesSnapshotTestJSON-1750456804 tempest-VolumesSnapshotTestJSON-1750456804-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:25:41.980063 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-f8c0b76d-6a1b-4705-b88c-902a59dc2bfa tempest-VolumesSnapshotTestJSON-1750456804 tempest-VolumesSnapshotTestJSON-1750456804-project] [instance: c31c26a8-fe5a-40fb-91a1-7b5251fd0daa] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:25:42.049869 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-26c3b219-6952-412b-b23d-bae773e54ed4 tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Calling method '>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:25:42.055005 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-f8c0b76d-6a1b-4705-b88c-902a59dc2bfa tempest-VolumesSnapshotTestJSON-1750456804 tempest-VolumesSnapshotTestJSON-1750456804-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '75744eccc3d24d4c9282cbe3a3314489', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fa43d54225bf44b283f9db5ddbbb13ca', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:25:42.056114 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-26c3b219-6952-412b-b23d-bae773e54ed4 tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] [instance: e49199b7-e998-44fb-9995-e6a13fd44dc6] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:25:42.064827 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-f8c0b76d-6a1b-4705-b88c-902a59dc2bfa tempest-VolumesSnapshotTestJSON-1750456804 tempest-VolumesSnapshotTestJSON-1750456804-project] 10.222.0.52 "GET /compute/v2.1/servers/c31c26a8-fe5a-40fb-91a1-7b5251fd0daa" status: 200 len: 1219 microversion: 2.1 time: 0.092294 Aug 27 02:25:42.065893 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 214/424] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:25:41 2022] GET /compute/v2.1/servers/c31c26a8-fe5a-40fb-91a1-7b5251fd0daa => generated 1219 bytes in 94 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:25:42.069270 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-26c3b219-6952-412b-b23d-bae773e54ed4 tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:25:42.070241 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-26c3b219-6952-412b-b23d-bae773e54ed4 tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:25:42.071114 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-26c3b219-6952-412b-b23d-bae773e54ed4 tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:25:42.072024 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-26c3b219-6952-412b-b23d-bae773e54ed4 tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:25:42.072566 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-26c3b219-6952-412b-b23d-bae773e54ed4 tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:25:42.073113 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-26c3b219-6952-412b-b23d-bae773e54ed4 tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:25:42.133804 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-26c3b219-6952-412b-b23d-bae773e54ed4 tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:25:42.134294 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-26c3b219-6952-412b-b23d-bae773e54ed4 tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:25:42.134727 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-26c3b219-6952-412b-b23d-bae773e54ed4 tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:25:42.135203 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.objects.instance [None req-26c3b219-6952-412b-b23d-bae773e54ed4 tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Lazy-loading 'fault' on Instance uuid e49199b7-e998-44fb-9995-e6a13fd44dc6 {{(pid=120864) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Aug 27 02:25:42.195231 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-26c3b219-6952-412b-b23d-bae773e54ed4 tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2e60cf2931f04f0e8e911adea32979dd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7cb3e3496c1a45c8942c3678c0c299e5', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:25:42.206448 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-26c3b219-6952-412b-b23d-bae773e54ed4 tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] 10.222.0.52 "GET /compute/v2.1/servers/e49199b7-e998-44fb-9995-e6a13fd44dc6" status: 200 len: 1303 microversion: 2.1 time: 0.162172 Aug 27 02:25:42.216817 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 211/425] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:25:42 2022] GET /compute/v2.1/servers/e49199b7-e998-44fb-9995-e6a13fd44dc6 => generated 1303 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 27 02:25:42.676554 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-eddcc604-2199-421a-88a0-2c7b1236ad75 tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:25:42.677682 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-eddcc604-2199-421a-88a0-2c7b1236ad75 tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] [instance: 9941c49f-8626-461d-a570-c0ffbfd4b804] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:25:42.684174 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-eddcc604-2199-421a-88a0-2c7b1236ad75 tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:25:42.684724 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-eddcc604-2199-421a-88a0-2c7b1236ad75 tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:25:42.685204 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-eddcc604-2199-421a-88a0-2c7b1236ad75 tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:25:42.685908 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-eddcc604-2199-421a-88a0-2c7b1236ad75 tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:25:42.686362 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-eddcc604-2199-421a-88a0-2c7b1236ad75 tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:25:42.686790 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-eddcc604-2199-421a-88a0-2c7b1236ad75 tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:25:42.839388 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [req-f6d3e7d1-fe59-4865-9747-789fcc9bf2ea req-ec5ea477-8edf-4c41-abc4-c2f72493e0e2 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "9941c49f-8626-461d-a570-c0ffbfd4b804", "tag": "89f4d982-1807-4e80-b9ef-c7bc0025f1b0"}]} {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 27 02:25:42.854285 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [req-f6d3e7d1-fe59-4865-9747-789fcc9bf2ea req-ec5ea477-8edf-4c41-abc4-c2f72493e0e2 service nova] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:25:42.855154 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [req-f6d3e7d1-fe59-4865-9747-789fcc9bf2ea req-ec5ea477-8edf-4c41-abc4-c2f72493e0e2 service nova] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:25:42.855813 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [req-f6d3e7d1-fe59-4865-9747-789fcc9bf2ea req-ec5ea477-8edf-4c41-abc4-c2f72493e0e2 service nova] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:25:42.874640 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.compute.server_external_events [req-f6d3e7d1-fe59-4865-9747-789fcc9bf2ea req-ec5ea477-8edf-4c41-abc4-c2f72493e0e2 service nova] Creating event network-changed:89f4d982-1807-4e80-b9ef-c7bc0025f1b0 for instance 9941c49f-8626-461d-a570-c0ffbfd4b804 on hv-cinder-84630 Aug 27 02:25:42.883566 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [req-f6d3e7d1-fe59-4865-9747-789fcc9bf2ea req-ec5ea477-8edf-4c41-abc4-c2f72493e0e2 service nova] 10.222.0.52 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.047729 Aug 27 02:25:42.884712 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 212/426] 10.222.0.52 () {64 vars in 1317 bytes} [Sat Aug 27 02:25:42 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Aug 27 02:25:42.892950 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-eddcc604-2199-421a-88a0-2c7b1236ad75 tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '642291c4daa04a6ab3fbce92a8be39c6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b2ddff853b7546788b645a0344c2297b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:25:42.901072 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-eddcc604-2199-421a-88a0-2c7b1236ad75 tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] 10.222.0.52 "GET /compute/v2.1/servers/9941c49f-8626-461d-a570-c0ffbfd4b804" status: 200 len: 1260 microversion: 2.1 time: 0.228746 Aug 27 02:25:42.901658 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 215/427] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:25:42 2022] GET /compute/v2.1/servers/9941c49f-8626-461d-a570-c0ffbfd4b804 => generated 1260 bytes in 230 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:25:42.963104 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-d6ddf91f-590f-4f66-80e8-e23214be6c2f tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Action: 'create', calling method: .version_select at 0x7f6b1b30c670>, body: {"keypair": {"name": "tempest-keypair-341970461", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBJr3vXYVSbmKuTSEqJyLksi2Ald5o9p9Fz+Zdt/4yQWfZVqw4lxH4M6m0gKcmN1I1fEv+oeAMTaGZuReh6G2A0qQnLURA6Q1UJnDY3wefyHc0cLIDLo9ZXJ7eehFxHctHw=="}} {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 27 02:25:42.980346 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.quota [None req-d6ddf91f-590f-4f66-80e8-e23214be6c2f tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Getting quotas for project 880ac4003248441c809a76cfd1b221eb. Resources: {'key_pairs'} {{(pid=120864) _get_quotas /opt/stack/nova/nova/quota.py:393}} Aug 27 02:25:42.984360 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.quota [None req-d6ddf91f-590f-4f66-80e8-e23214be6c2f tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Getting quotas for user d945d512ae034ec08838d0a3afde2429 and project 880ac4003248441c809a76cfd1b221eb. Resources: {'key_pairs'} {{(pid=120864) _get_quotas /opt/stack/nova/nova/quota.py:383}} Aug 27 02:25:43.010688 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-d6ddf91f-590f-4f66-80e8-e23214be6c2f tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] 10.222.0.52 "POST /compute/v2.1/os-keypairs" status: 200 len: 385 microversion: 2.1 time: 0.052208 Aug 27 02:25:43.011276 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 213/428] 10.222.0.52 () {60 vars in 1159 bytes} [Sat Aug 27 02:25:42 2022] POST /compute/v2.1/os-keypairs => generated 385 bytes in 54 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Aug 27 02:25:43.086216 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-681c15ec-05b2-4ac3-b4bc-adb7863f5635 tempest-VolumesSnapshotTestJSON-1750456804 tempest-VolumesSnapshotTestJSON-1750456804-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:25:43.086828 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-681c15ec-05b2-4ac3-b4bc-adb7863f5635 tempest-VolumesSnapshotTestJSON-1750456804 tempest-VolumesSnapshotTestJSON-1750456804-project] [instance: c31c26a8-fe5a-40fb-91a1-7b5251fd0daa] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:25:43.095520 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-681c15ec-05b2-4ac3-b4bc-adb7863f5635 tempest-VolumesSnapshotTestJSON-1750456804 tempest-VolumesSnapshotTestJSON-1750456804-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:25:43.096010 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-681c15ec-05b2-4ac3-b4bc-adb7863f5635 tempest-VolumesSnapshotTestJSON-1750456804 tempest-VolumesSnapshotTestJSON-1750456804-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:25:43.096440 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-681c15ec-05b2-4ac3-b4bc-adb7863f5635 tempest-VolumesSnapshotTestJSON-1750456804 tempest-VolumesSnapshotTestJSON-1750456804-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:25:43.097225 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-681c15ec-05b2-4ac3-b4bc-adb7863f5635 tempest-VolumesSnapshotTestJSON-1750456804 tempest-VolumesSnapshotTestJSON-1750456804-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:25:43.098163 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-681c15ec-05b2-4ac3-b4bc-adb7863f5635 tempest-VolumesSnapshotTestJSON-1750456804 tempest-VolumesSnapshotTestJSON-1750456804-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:25:43.098284 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-681c15ec-05b2-4ac3-b4bc-adb7863f5635 tempest-VolumesSnapshotTestJSON-1750456804 tempest-VolumesSnapshotTestJSON-1750456804-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:25:43.173140 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-681c15ec-05b2-4ac3-b4bc-adb7863f5635 tempest-VolumesSnapshotTestJSON-1750456804 tempest-VolumesSnapshotTestJSON-1750456804-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '75744eccc3d24d4c9282cbe3a3314489', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fa43d54225bf44b283f9db5ddbbb13ca', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:25:43.182347 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-681c15ec-05b2-4ac3-b4bc-adb7863f5635 tempest-VolumesSnapshotTestJSON-1750456804 tempest-VolumesSnapshotTestJSON-1750456804-project] 10.222.0.52 "GET /compute/v2.1/servers/c31c26a8-fe5a-40fb-91a1-7b5251fd0daa" status: 200 len: 1271 microversion: 2.1 time: 0.099916 Aug 27 02:25:43.182923 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 216/429] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:25:43 2022] GET /compute/v2.1/servers/c31c26a8-fe5a-40fb-91a1-7b5251fd0daa => generated 1271 bytes in 101 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 27 02:25:43.228644 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-4ea4315b-c89d-42d5-b22c-93a646022d04 tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Calling method '>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:25:43.229406 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-4ea4315b-c89d-42d5-b22c-93a646022d04 tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] [instance: e49199b7-e998-44fb-9995-e6a13fd44dc6] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:25:43.236058 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-4ea4315b-c89d-42d5-b22c-93a646022d04 tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:25:43.236531 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-4ea4315b-c89d-42d5-b22c-93a646022d04 tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:25:43.236977 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-4ea4315b-c89d-42d5-b22c-93a646022d04 tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:25:43.237691 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-4ea4315b-c89d-42d5-b22c-93a646022d04 tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:25:43.238341 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-4ea4315b-c89d-42d5-b22c-93a646022d04 tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:25:43.238861 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-4ea4315b-c89d-42d5-b22c-93a646022d04 tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:25:43.252140 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.wsgi [None req-4ea4315b-c89d-42d5-b22c-93a646022d04 tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] HTTP exception thrown: Instance e49199b7-e998-44fb-9995-e6a13fd44dc6 could not be found. Aug 27 02:25:43.253982 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-4ea4315b-c89d-42d5-b22c-93a646022d04 tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Request method failure captured: Aug 27 02:25:43.253982 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: request: GET /compute/v2.1/servers/e49199b7-e998-44fb-9995-e6a13fd44dc6 HTTP/1.1 Aug 27 02:25:43.253982 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: Accept: application/json Aug 27 02:25:43.253982 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: Accept-Encoding: identity Aug 27 02:25:43.253982 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: Connection: close Aug 27 02:25:43.253982 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: Content-Length: 0 Aug 27 02:25:43.253982 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: Content-Type: application/json Aug 27 02:25:43.253982 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: Host: 10.222.0.52 Aug 27 02:25:43.253982 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: Openstack-System-Scope: None Aug 27 02:25:43.253982 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: User-Agent: python-urllib3/1.26.11 Aug 27 02:25:43.253982 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-Auth-Token: gAAAAABjCYB0MyJqOfE5kKRO8uoWsCg0klSjiu7lxuQEUldbXVIPAZZ5yV6_hhSVHubtYc4X9p9JyWfm8mZ1gFcYpJGdJ8Ek7ubkjfN8IvIv2cpKCCIp1zSQKzKaTdPuvOUqwCXR64qVlUgO_Kp5x1BMLZilD49VGKqBjblPsc1kPAfNx47Clds Aug 27 02:25:43.253982 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-Domain-Id: None Aug 27 02:25:43.253982 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-Domain-Name: None Aug 27 02:25:43.253982 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-Identity-Status: Confirmed Aug 27 02:25:43.253982 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-Is-Admin-Project: True Aug 27 02:25:43.253982 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-Project-Domain-Id: default Aug 27 02:25:43.253982 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-Project-Domain-Name: Default Aug 27 02:25:43.253982 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-Project-Id: 7cb3e3496c1a45c8942c3678c0c299e5 Aug 27 02:25:43.253982 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-Project-Name: tempest-VolumesActionsTest-477223312 Aug 27 02:25:43.253982 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-Role: reader,member Aug 27 02:25:43.253982 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-Roles: reader,member Aug 27 02:25:43.255448 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-Service-Catalog: [{"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.52:8080", "publicURL": "http://10.222.0.52:8080/v1/AUTH_7cb3e3496c1a45c8942c3678c0c299e5"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52:9696/networking"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/compute/v2/7cb3e3496c1a45c8942c3678c0c299e5"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/image"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/volume/v3/7cb3e3496c1a45c8942c3678c0c299e5"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/identity"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/volume/v3/7cb3e3496c1a45c8942c3678c0c299e5"}]}] Aug 27 02:25:43.255448 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-Tenant: tempest-VolumesActionsTest-477223312 Aug 27 02:25:43.255448 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-Tenant-Id: 7cb3e3496c1a45c8942c3678c0c299e5 Aug 27 02:25:43.255448 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-Tenant-Name: tempest-VolumesActionsTest-477223312 Aug 27 02:25:43.255448 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-User: tempest-VolumesActionsTest-477223312-project Aug 27 02:25:43.255448 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-User-Domain-Id: default Aug 27 02:25:43.255448 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-User-Domain-Name: Default Aug 27 02:25:43.255448 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-User-Id: 2e60cf2931f04f0e8e911adea32979dd Aug 27 02:25:43.255448 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-User-Name: tempest-VolumesActionsTest-477223312-project Aug 27 02:25:43.255448 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: method: > Aug 27 02:25:43.255448 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: exception: Instance e49199b7-e998-44fb-9995-e6a13fd44dc6 could not be found. Aug 27 02:25:43.255448 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: action_args: {'id': 'e49199b7-e998-44fb-9995-e6a13fd44dc6'} Aug 27 02:25:43.256345 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Aug 27 02:25:43.256345 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 27 02:25:43.256345 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Aug 27 02:25:43.256345 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Aug 27 02:25:43.256345 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Aug 27 02:25:43.256345 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Aug 27 02:25:43.256345 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Aug 27 02:25:43.256345 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Aug 27 02:25:43.256345 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Aug 27 02:25:43.256345 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Aug 27 02:25:43.256345 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance e49199b7-e998-44fb-9995-e6a13fd44dc6 could not be found. Aug 27 02:25:43.256345 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi Aug 27 02:25:43.256345 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 27 02:25:43.256345 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi Aug 27 02:25:43.256345 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 27 02:25:43.256345 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 27 02:25:43.256345 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 27 02:25:43.256345 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Aug 27 02:25:43.256345 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Aug 27 02:25:43.256345 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Aug 27 02:25:43.257703 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Aug 27 02:25:43.257703 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Aug 27 02:25:43.257703 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi instance = self._get_server( Aug 27 02:25:43.257703 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Aug 27 02:25:43.257703 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Aug 27 02:25:43.257703 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Aug 27 02:25:43.257703 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Aug 27 02:25:43.257703 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance e49199b7-e998-44fb-9995-e6a13fd44dc6 could not be found. Aug 27 02:25:43.257703 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi Aug 27 02:25:43.257703 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 27 02:25:43.257703 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi Aug 27 02:25:43.257703 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 27 02:25:43.257703 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 27 02:25:43.257703 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 27 02:25:43.257703 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Aug 27 02:25:43.257703 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Aug 27 02:25:43.257703 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance e49199b7-e998-44fb-9995-e6a13fd44dc6 could not be found. Aug 27 02:25:43.257703 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi Aug 27 02:25:43.258823 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-4ea4315b-c89d-42d5-b22c-93a646022d04 tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Returning 404 to user: Instance e49199b7-e998-44fb-9995-e6a13fd44dc6 could not be found. {{(pid=120864) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Aug 27 02:25:43.258823 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-4ea4315b-c89d-42d5-b22c-93a646022d04 tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] 10.222.0.52 "GET /compute/v2.1/servers/e49199b7-e998-44fb-9995-e6a13fd44dc6" status: 404 len: 111 microversion: 2.1 time: 0.030632 Aug 27 02:25:43.259608 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 214/430] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:25:43 2022] GET /compute/v2.1/servers/e49199b7-e998-44fb-9995-e6a13fd44dc6 => generated 111 bytes in 36 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Aug 27 02:25:43.379267 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-0a1ee10d-2f75-409c-8826-e77656a92a2c tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-VolumesActionsTest-instance-2020382085", "imageRef": "9fc30b36-b827-46f9-892d-30e808bc2b8a", "flavorRef": "452", "networks": [{"uuid": "d622f246-857a-45c7-a4ec-c30529f7e62e"}]}} {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 27 02:25:43.445037 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.network.neutron [None req-0a1ee10d-2f75-409c-8826-e77656a92a2c tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] validate_networks() for [('d622f246-857a-45c7-a4ec-c30529f7e62e', None, None, None, None, None)] {{(pid=120863) validate_networks /opt/stack/nova/nova/network/neutron.py:2626}} Aug 27 02:25:43.923299 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-cd2ad4aa-6e7f-45db-9583-95fb63f4e2c1 tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] Calling method '>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:25:43.923848 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-cd2ad4aa-6e7f-45db-9583-95fb63f4e2c1 tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] [instance: 9941c49f-8626-461d-a570-c0ffbfd4b804] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:25:43.931860 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-cd2ad4aa-6e7f-45db-9583-95fb63f4e2c1 tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:25:43.932536 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-cd2ad4aa-6e7f-45db-9583-95fb63f4e2c1 tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:25:43.932772 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-cd2ad4aa-6e7f-45db-9583-95fb63f4e2c1 tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:25:43.933730 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-cd2ad4aa-6e7f-45db-9583-95fb63f4e2c1 tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:25:43.934298 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-cd2ad4aa-6e7f-45db-9583-95fb63f4e2c1 tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:25:43.934649 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-cd2ad4aa-6e7f-45db-9583-95fb63f4e2c1 tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:25:44.269223 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-cd2ad4aa-6e7f-45db-9583-95fb63f4e2c1 tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '642291c4daa04a6ab3fbce92a8be39c6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b2ddff853b7546788b645a0344c2297b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:25:44.277369 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-cd2ad4aa-6e7f-45db-9583-95fb63f4e2c1 tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] 10.222.0.52 "GET /compute/v2.1/servers/9941c49f-8626-461d-a570-c0ffbfd4b804" status: 200 len: 1260 microversion: 2.1 time: 0.357903 Aug 27 02:25:44.278017 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 215/431] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:25:43 2022] GET /compute/v2.1/servers/9941c49f-8626-461d-a570-c0ffbfd4b804 => generated 1260 bytes in 359 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:25:44.282651 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-3309cb57-38ec-4610-8e77-3f7958cce302 tempest-VolumesSnapshotTestJSON-1750456804 tempest-VolumesSnapshotTestJSON-1750456804-project] Calling method '>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:25:44.283242 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-3309cb57-38ec-4610-8e77-3f7958cce302 tempest-VolumesSnapshotTestJSON-1750456804 tempest-VolumesSnapshotTestJSON-1750456804-project] [instance: c31c26a8-fe5a-40fb-91a1-7b5251fd0daa] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:25:44.291543 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-3309cb57-38ec-4610-8e77-3f7958cce302 tempest-VolumesSnapshotTestJSON-1750456804 tempest-VolumesSnapshotTestJSON-1750456804-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:25:44.292089 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-3309cb57-38ec-4610-8e77-3f7958cce302 tempest-VolumesSnapshotTestJSON-1750456804 tempest-VolumesSnapshotTestJSON-1750456804-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:25:44.292497 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-3309cb57-38ec-4610-8e77-3f7958cce302 tempest-VolumesSnapshotTestJSON-1750456804 tempest-VolumesSnapshotTestJSON-1750456804-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:25:44.293271 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-3309cb57-38ec-4610-8e77-3f7958cce302 tempest-VolumesSnapshotTestJSON-1750456804 tempest-VolumesSnapshotTestJSON-1750456804-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:25:44.293892 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-3309cb57-38ec-4610-8e77-3f7958cce302 tempest-VolumesSnapshotTestJSON-1750456804 tempest-VolumesSnapshotTestJSON-1750456804-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:25:44.294312 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-3309cb57-38ec-4610-8e77-3f7958cce302 tempest-VolumesSnapshotTestJSON-1750456804 tempest-VolumesSnapshotTestJSON-1750456804-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:25:44.410303 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-3309cb57-38ec-4610-8e77-3f7958cce302 tempest-VolumesSnapshotTestJSON-1750456804 tempest-VolumesSnapshotTestJSON-1750456804-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '75744eccc3d24d4c9282cbe3a3314489', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fa43d54225bf44b283f9db5ddbbb13ca', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:25:44.419143 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-3309cb57-38ec-4610-8e77-3f7958cce302 tempest-VolumesSnapshotTestJSON-1750456804 tempest-VolumesSnapshotTestJSON-1750456804-project] 10.222.0.52 "GET /compute/v2.1/servers/c31c26a8-fe5a-40fb-91a1-7b5251fd0daa" status: 200 len: 1277 microversion: 2.1 time: 0.140065 Aug 27 02:25:44.419960 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 216/432] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:25:44 2022] GET /compute/v2.1/servers/c31c26a8-fe5a-40fb-91a1-7b5251fd0daa => generated 1277 bytes in 141 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:25:44.500890 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.virt.hardware [None req-0a1ee10d-2f75-409c-8826-e77656a92a2c tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Flavor limits 0:0:0 {{(pid=120863) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Aug 27 02:25:44.501629 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.virt.hardware [None req-0a1ee10d-2f75-409c-8826-e77656a92a2c tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Image limits 0:0:0 {{(pid=120863) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Aug 27 02:25:44.501933 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.virt.hardware [None req-0a1ee10d-2f75-409c-8826-e77656a92a2c tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Flavor pref 0:0:0 {{(pid=120863) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Aug 27 02:25:44.502316 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.virt.hardware [None req-0a1ee10d-2f75-409c-8826-e77656a92a2c tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Image pref 0:0:0 {{(pid=120863) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Aug 27 02:25:44.502963 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.virt.hardware [None req-0a1ee10d-2f75-409c-8826-e77656a92a2c tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=120863) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Aug 27 02:25:44.512707 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-0a1ee10d-2f75-409c-8826-e77656a92a2c tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:25:44.513223 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-0a1ee10d-2f75-409c-8826-e77656a92a2c tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:25:44.513830 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-0a1ee10d-2f75-409c-8826-e77656a92a2c tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:25:44.514548 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-0a1ee10d-2f75-409c-8826-e77656a92a2c tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:25:44.514995 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-0a1ee10d-2f75-409c-8826-e77656a92a2c tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:25:44.515415 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-0a1ee10d-2f75-409c-8826-e77656a92a2c tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:25:44.533048 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.quota [None req-0a1ee10d-2f75-409c-8826-e77656a92a2c tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Getting quotas for project 7cb3e3496c1a45c8942c3678c0c299e5. Resources: {'cores', 'ram', 'instances'} {{(pid=120863) _get_quotas /opt/stack/nova/nova/quota.py:393}} Aug 27 02:25:44.538545 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.quota [None req-0a1ee10d-2f75-409c-8826-e77656a92a2c tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Getting quotas for user 2e60cf2931f04f0e8e911adea32979dd and project 7cb3e3496c1a45c8942c3678c0c299e5. Resources: {'cores', 'ram', 'instances'} {{(pid=120863) _get_quotas /opt/stack/nova/nova/quota.py:383}} Aug 27 02:25:44.550762 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-0a1ee10d-2f75-409c-8826-e77656a92a2c tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Going to run 1 instances... {{(pid=120863) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Aug 27 02:25:44.553287 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-0a1ee10d-2f75-409c-8826-e77656a92a2c tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] [instance: fbea9949-6c55-4ae7-96e1-b09f1a8030f1] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='9fc30b36-b827-46f9-892d-30e808bc2b8a',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=120863) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Aug 27 02:25:44.600971 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR oslo.messaging._drivers.impl_rabbit [None req-0a1ee10d-2f75-409c-8826-e77656a92a2c tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] [eff717f0-00d1-4b46-8f37-d668394dce5f] AMQP server on 10.222.0.52:5672 is unreachable: [Errno 104] Connection reset by peer. Trying again in 1 seconds.: ConnectionResetError: [Errno 104] Connection reset by peer Aug 27 02:25:45.282860 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-59cf490b-027c-4549-9300-e43ea5521c74 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-ServerRescueNegativeTestJSON-server-600312448", "imageRef": "9fc30b36-b827-46f9-892d-30e808bc2b8a", "flavorRef": "452", "adminPass": "***", "networks": [{"uuid": "8ba10e7b-62f2-400d-a225-3160ea7a308c"}], "security_groups": [{"name": "tempest-securitygroup--1458706136"}], "key_name": "tempest-keypair-341970461", "user_data": "}} {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 27 02:25:45.470598 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.network.neutron [None req-59cf490b-027c-4549-9300-e43ea5521c74 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] validate_networks() for [('8ba10e7b-62f2-400d-a225-3160ea7a308c', None, None, None, None, None)] {{(pid=120864) validate_networks /opt/stack/nova/nova/network/neutron.py:2626}} Aug 27 02:25:45.609876 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO oslo.messaging._drivers.impl_rabbit [None req-0a1ee10d-2f75-409c-8826-e77656a92a2c tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] [eff717f0-00d1-4b46-8f37-d668394dce5f] Reconnected to AMQP server on 10.222.0.52:5672 via [amqp] client with port 43368. Aug 27 02:25:45.614084 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-0a1ee10d-2f75-409c-8826-e77656a92a2c tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] 10.222.0.52 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 2.239331 Aug 27 02:25:45.614957 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 217/433] 10.222.0.52 () {60 vars in 1147 bytes} [Sat Aug 27 02:25:43 2022] POST /compute/v2.1/servers => generated 380 bytes in 2241 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Aug 27 02:25:45.619781 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-93713361-76a4-4db3-9d6e-c7f9a48ab5ca tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:25:45.620766 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-93713361-76a4-4db3-9d6e-c7f9a48ab5ca tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] [instance: 9941c49f-8626-461d-a570-c0ffbfd4b804] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:25:45.628010 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-93713361-76a4-4db3-9d6e-c7f9a48ab5ca tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:25:45.628872 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-93713361-76a4-4db3-9d6e-c7f9a48ab5ca tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:25:45.629560 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-93713361-76a4-4db3-9d6e-c7f9a48ab5ca tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:25:45.630581 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-93713361-76a4-4db3-9d6e-c7f9a48ab5ca tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:25:45.631190 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-93713361-76a4-4db3-9d6e-c7f9a48ab5ca tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:25:45.631771 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-93713361-76a4-4db3-9d6e-c7f9a48ab5ca tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:25:46.030608 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-93713361-76a4-4db3-9d6e-c7f9a48ab5ca tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '642291c4daa04a6ab3fbce92a8be39c6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b2ddff853b7546788b645a0344c2297b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:25:46.038617 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-93713361-76a4-4db3-9d6e-c7f9a48ab5ca tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] 10.222.0.52 "GET /compute/v2.1/servers/9941c49f-8626-461d-a570-c0ffbfd4b804" status: 200 len: 1260 microversion: 2.1 time: 0.422473 Aug 27 02:25:46.039377 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 218/434] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:25:45 2022] GET /compute/v2.1/servers/9941c49f-8626-461d-a570-c0ffbfd4b804 => generated 1260 bytes in 424 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:25:46.044978 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-bf580f69-9253-46f8-9721-ab962ada8a2b tempest-VolumesSnapshotTestJSON-1750456804 tempest-VolumesSnapshotTestJSON-1750456804-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:25:46.045855 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-bf580f69-9253-46f8-9721-ab962ada8a2b tempest-VolumesSnapshotTestJSON-1750456804 tempest-VolumesSnapshotTestJSON-1750456804-project] [instance: c31c26a8-fe5a-40fb-91a1-7b5251fd0daa] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:25:46.053744 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-bf580f69-9253-46f8-9721-ab962ada8a2b tempest-VolumesSnapshotTestJSON-1750456804 tempest-VolumesSnapshotTestJSON-1750456804-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:25:46.054694 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-bf580f69-9253-46f8-9721-ab962ada8a2b tempest-VolumesSnapshotTestJSON-1750456804 tempest-VolumesSnapshotTestJSON-1750456804-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:25:46.055133 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-bf580f69-9253-46f8-9721-ab962ada8a2b tempest-VolumesSnapshotTestJSON-1750456804 tempest-VolumesSnapshotTestJSON-1750456804-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:25:46.055835 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-bf580f69-9253-46f8-9721-ab962ada8a2b tempest-VolumesSnapshotTestJSON-1750456804 tempest-VolumesSnapshotTestJSON-1750456804-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:25:46.056371 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-bf580f69-9253-46f8-9721-ab962ada8a2b tempest-VolumesSnapshotTestJSON-1750456804 tempest-VolumesSnapshotTestJSON-1750456804-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:25:46.056791 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-bf580f69-9253-46f8-9721-ab962ada8a2b tempest-VolumesSnapshotTestJSON-1750456804 tempest-VolumesSnapshotTestJSON-1750456804-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:25:46.250364 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.virt.hardware [None req-59cf490b-027c-4549-9300-e43ea5521c74 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Flavor limits 0:0:0 {{(pid=120864) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Aug 27 02:25:46.251249 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.virt.hardware [None req-59cf490b-027c-4549-9300-e43ea5521c74 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Image limits 0:0:0 {{(pid=120864) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Aug 27 02:25:46.251786 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.virt.hardware [None req-59cf490b-027c-4549-9300-e43ea5521c74 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Flavor pref 0:0:0 {{(pid=120864) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Aug 27 02:25:46.252158 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.virt.hardware [None req-59cf490b-027c-4549-9300-e43ea5521c74 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Image pref 0:0:0 {{(pid=120864) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Aug 27 02:25:46.252736 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.virt.hardware [None req-59cf490b-027c-4549-9300-e43ea5521c74 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=120864) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Aug 27 02:25:46.263092 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-59cf490b-027c-4549-9300-e43ea5521c74 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:25:46.263550 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-59cf490b-027c-4549-9300-e43ea5521c74 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:25:46.263989 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-59cf490b-027c-4549-9300-e43ea5521c74 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:25:46.264926 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-59cf490b-027c-4549-9300-e43ea5521c74 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:25:46.265398 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-59cf490b-027c-4549-9300-e43ea5521c74 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:25:46.265795 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-59cf490b-027c-4549-9300-e43ea5521c74 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:25:46.286101 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.quota [None req-59cf490b-027c-4549-9300-e43ea5521c74 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Getting quotas for project 880ac4003248441c809a76cfd1b221eb. Resources: {'cores', 'ram', 'instances'} {{(pid=120864) _get_quotas /opt/stack/nova/nova/quota.py:393}} Aug 27 02:25:46.296867 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.quota [None req-59cf490b-027c-4549-9300-e43ea5521c74 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Getting quotas for user d945d512ae034ec08838d0a3afde2429 and project 880ac4003248441c809a76cfd1b221eb. Resources: {'cores', 'ram', 'instances'} {{(pid=120864) _get_quotas /opt/stack/nova/nova/quota.py:383}} Aug 27 02:25:46.310122 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-59cf490b-027c-4549-9300-e43ea5521c74 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Going to run 1 instances... {{(pid=120864) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Aug 27 02:25:46.312376 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-59cf490b-027c-4549-9300-e43ea5521c74 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] [instance: 98e4cfeb-d49e-4888-a2c0-40a9e8af90d3] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='9fc30b36-b827-46f9-892d-30e808bc2b8a',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=120864) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Aug 27 02:25:46.323383 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG dbcounter [-] [120864] Writing DB stats nova_api:SELECT=92,nova_api:UPDATE=1,nova_api:INSERT=7 {{(pid=120864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 27 02:25:46.345362 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-bf580f69-9253-46f8-9721-ab962ada8a2b tempest-VolumesSnapshotTestJSON-1750456804 tempest-VolumesSnapshotTestJSON-1750456804-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '75744eccc3d24d4c9282cbe3a3314489', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fa43d54225bf44b283f9db5ddbbb13ca', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:25:46.361329 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-bf580f69-9253-46f8-9721-ab962ada8a2b tempest-VolumesSnapshotTestJSON-1750456804 tempest-VolumesSnapshotTestJSON-1750456804-project] 10.222.0.52 "GET /compute/v2.1/servers/c31c26a8-fe5a-40fb-91a1-7b5251fd0daa" status: 200 len: 1319 microversion: 2.1 time: 0.320693 Aug 27 02:25:46.364252 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-59cf490b-027c-4549-9300-e43ea5521c74 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] 10.222.0.52 "POST /compute/v2.1/servers" status: 202 len: 409 microversion: 2.1 time: 1.085217 Aug 27 02:25:46.364935 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 217/435] 10.222.0.52 () {60 vars in 1147 bytes} [Sat Aug 27 02:25:45 2022] POST /compute/v2.1/servers => generated 409 bytes in 1087 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Aug 27 02:25:46.369758 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 219/436] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:25:46 2022] GET /compute/v2.1/servers/c31c26a8-fe5a-40fb-91a1-7b5251fd0daa => generated 1319 bytes in 330 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:25:46.369905 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-01c4f426-ef50-4092-8ce7-9ef1f083a43c tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Calling method '>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:25:46.370362 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-01c4f426-ef50-4092-8ce7-9ef1f083a43c tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] [instance: fbea9949-6c55-4ae7-96e1-b09f1a8030f1] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:25:46.377478 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-01c4f426-ef50-4092-8ce7-9ef1f083a43c tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:25:46.378199 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-01c4f426-ef50-4092-8ce7-9ef1f083a43c tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:25:46.378828 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-01c4f426-ef50-4092-8ce7-9ef1f083a43c tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:25:46.379667 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-01c4f426-ef50-4092-8ce7-9ef1f083a43c tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:25:46.380258 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-01c4f426-ef50-4092-8ce7-9ef1f083a43c tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:25:46.380872 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-01c4f426-ef50-4092-8ce7-9ef1f083a43c tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:25:46.390438 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-ee91f73a-1745-4d2f-b4c7-df8f67e4702b tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:25:46.416335 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-ee91f73a-1745-4d2f-b4c7-df8f67e4702b tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] [instance: 98e4cfeb-d49e-4888-a2c0-40a9e8af90d3] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:25:46.469883 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-01c4f426-ef50-4092-8ce7-9ef1f083a43c tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2e60cf2931f04f0e8e911adea32979dd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7cb3e3496c1a45c8942c3678c0c299e5', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:25:46.480225 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-01c4f426-ef50-4092-8ce7-9ef1f083a43c tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] 10.222.0.52 "GET /compute/v2.1/servers/fbea9949-6c55-4ae7-96e1-b09f1a8030f1" status: 200 len: 1266 microversion: 2.1 time: 0.114152 Aug 27 02:25:46.480797 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 218/437] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:25:46 2022] GET /compute/v2.1/servers/fbea9949-6c55-4ae7-96e1-b09f1a8030f1 => generated 1266 bytes in 115 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:25:46.502891 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-ee91f73a-1745-4d2f-b4c7-df8f67e4702b tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd945d512ae034ec08838d0a3afde2429', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '880ac4003248441c809a76cfd1b221eb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:25:46.513471 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-ee91f73a-1745-4d2f-b4c7-df8f67e4702b tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] 10.222.0.52 "GET /compute/v2.1/servers/98e4cfeb-d49e-4888-a2c0-40a9e8af90d3" status: 200 len: 1244 microversion: 2.1 time: 0.130867 Aug 27 02:25:46.514231 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 220/438] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:25:46 2022] GET /compute/v2.1/servers/98e4cfeb-d49e-4888-a2c0-40a9e8af90d3 => generated 1244 bytes in 132 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:25:47.060754 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-9137f519-087a-4a19-a6c0-8649cb32d890 tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] Calling method '>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:25:47.061982 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-9137f519-087a-4a19-a6c0-8649cb32d890 tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] [instance: 9941c49f-8626-461d-a570-c0ffbfd4b804] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:25:47.074254 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-9137f519-087a-4a19-a6c0-8649cb32d890 tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:25:47.074748 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-9137f519-087a-4a19-a6c0-8649cb32d890 tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:25:47.075355 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-9137f519-087a-4a19-a6c0-8649cb32d890 tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:25:47.076384 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-9137f519-087a-4a19-a6c0-8649cb32d890 tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:25:47.076819 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-9137f519-087a-4a19-a6c0-8649cb32d890 tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:25:47.077272 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-9137f519-087a-4a19-a6c0-8649cb32d890 tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:25:47.094266 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [req-4dd8b06d-62c6-4572-b9e9-5abb685e8d58 req-c7b8146e-b3f5-48d9-80db-9a8232432875 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "c31c26a8-fe5a-40fb-91a1-7b5251fd0daa", "tag": "22831215-7029-41bf-a558-6aa1056c1015"}]} {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 27 02:25:47.106213 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [req-4dd8b06d-62c6-4572-b9e9-5abb685e8d58 req-c7b8146e-b3f5-48d9-80db-9a8232432875 service nova] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:25:47.107128 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [req-4dd8b06d-62c6-4572-b9e9-5abb685e8d58 req-c7b8146e-b3f5-48d9-80db-9a8232432875 service nova] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:25:47.107948 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [req-4dd8b06d-62c6-4572-b9e9-5abb685e8d58 req-c7b8146e-b3f5-48d9-80db-9a8232432875 service nova] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:25:47.124944 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG dbcounter [-] [120863] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=120863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 27 02:25:47.127960 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.compute.server_external_events [req-4dd8b06d-62c6-4572-b9e9-5abb685e8d58 req-c7b8146e-b3f5-48d9-80db-9a8232432875 service nova] Creating event network-changed:22831215-7029-41bf-a558-6aa1056c1015 for instance c31c26a8-fe5a-40fb-91a1-7b5251fd0daa on hv-cinder-84630 Aug 27 02:25:47.134871 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [req-4dd8b06d-62c6-4572-b9e9-5abb685e8d58 req-c7b8146e-b3f5-48d9-80db-9a8232432875 service nova] 10.222.0.52 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.044146 Aug 27 02:25:47.135819 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 221/439] 10.222.0.52 () {64 vars in 1317 bytes} [Sat Aug 27 02:25:47 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Aug 27 02:25:47.317889 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-9137f519-087a-4a19-a6c0-8649cb32d890 tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '642291c4daa04a6ab3fbce92a8be39c6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b2ddff853b7546788b645a0344c2297b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:25:47.327944 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-9137f519-087a-4a19-a6c0-8649cb32d890 tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] 10.222.0.52 "GET /compute/v2.1/servers/9941c49f-8626-461d-a570-c0ffbfd4b804" status: 200 len: 1260 microversion: 2.1 time: 0.271230 Aug 27 02:25:47.328505 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 219/440] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:25:47 2022] GET /compute/v2.1/servers/9941c49f-8626-461d-a570-c0ffbfd4b804 => generated 1260 bytes in 273 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:25:47.396419 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-bcd13186-fa3b-484d-a800-75194f943c95 tempest-VolumesSnapshotTestJSON-1750456804 tempest-VolumesSnapshotTestJSON-1750456804-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:25:47.397185 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-bcd13186-fa3b-484d-a800-75194f943c95 tempest-VolumesSnapshotTestJSON-1750456804 tempest-VolumesSnapshotTestJSON-1750456804-project] [instance: c31c26a8-fe5a-40fb-91a1-7b5251fd0daa] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:25:47.404407 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-bcd13186-fa3b-484d-a800-75194f943c95 tempest-VolumesSnapshotTestJSON-1750456804 tempest-VolumesSnapshotTestJSON-1750456804-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:25:47.404956 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-bcd13186-fa3b-484d-a800-75194f943c95 tempest-VolumesSnapshotTestJSON-1750456804 tempest-VolumesSnapshotTestJSON-1750456804-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:25:47.405458 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-bcd13186-fa3b-484d-a800-75194f943c95 tempest-VolumesSnapshotTestJSON-1750456804 tempest-VolumesSnapshotTestJSON-1750456804-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:25:47.406151 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-bcd13186-fa3b-484d-a800-75194f943c95 tempest-VolumesSnapshotTestJSON-1750456804 tempest-VolumesSnapshotTestJSON-1750456804-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:25:47.406602 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-bcd13186-fa3b-484d-a800-75194f943c95 tempest-VolumesSnapshotTestJSON-1750456804 tempest-VolumesSnapshotTestJSON-1750456804-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:25:47.407013 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-bcd13186-fa3b-484d-a800-75194f943c95 tempest-VolumesSnapshotTestJSON-1750456804 tempest-VolumesSnapshotTestJSON-1750456804-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:25:47.504034 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-b6b0b1b8-9159-4ef3-a731-a75186a492e0 tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Calling method '>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:25:47.505454 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-b6b0b1b8-9159-4ef3-a731-a75186a492e0 tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] [instance: fbea9949-6c55-4ae7-96e1-b09f1a8030f1] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:25:47.521343 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-b6b0b1b8-9159-4ef3-a731-a75186a492e0 tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:25:47.521823 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-b6b0b1b8-9159-4ef3-a731-a75186a492e0 tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:25:47.523899 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-b6b0b1b8-9159-4ef3-a731-a75186a492e0 tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:25:47.526100 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-b6b0b1b8-9159-4ef3-a731-a75186a492e0 tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:25:47.526773 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-b6b0b1b8-9159-4ef3-a731-a75186a492e0 tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:25:47.527429 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-b6b0b1b8-9159-4ef3-a731-a75186a492e0 tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:25:47.608114 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-b6b0b1b8-9159-4ef3-a731-a75186a492e0 tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2e60cf2931f04f0e8e911adea32979dd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7cb3e3496c1a45c8942c3678c0c299e5', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:25:47.615979 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-b6b0b1b8-9159-4ef3-a731-a75186a492e0 tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] 10.222.0.52 "GET /compute/v2.1/servers/fbea9949-6c55-4ae7-96e1-b09f1a8030f1" status: 200 len: 1272 microversion: 2.1 time: 0.118140 Aug 27 02:25:47.629144 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-bcd13186-fa3b-484d-a800-75194f943c95 tempest-VolumesSnapshotTestJSON-1750456804 tempest-VolumesSnapshotTestJSON-1750456804-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '75744eccc3d24d4c9282cbe3a3314489', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fa43d54225bf44b283f9db5ddbbb13ca', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:25:47.629144 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-bcd13186-fa3b-484d-a800-75194f943c95 tempest-VolumesSnapshotTestJSON-1750456804 tempest-VolumesSnapshotTestJSON-1750456804-project] 10.222.0.52 "GET /compute/v2.1/servers/c31c26a8-fe5a-40fb-91a1-7b5251fd0daa" status: 200 len: 1319 microversion: 2.1 time: 0.235752 Aug 27 02:25:47.629144 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 222/441] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:25:47 2022] GET /compute/v2.1/servers/c31c26a8-fe5a-40fb-91a1-7b5251fd0daa => generated 1319 bytes in 237 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:25:47.630266 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 220/442] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:25:47 2022] GET /compute/v2.1/servers/fbea9949-6c55-4ae7-96e1-b09f1a8030f1 => generated 1272 bytes in 133 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:25:47.642645 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-7d37d71e-e57f-4dda-a4a1-d8b9b04e5dec tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:25:47.643367 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-7d37d71e-e57f-4dda-a4a1-d8b9b04e5dec tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] [instance: 98e4cfeb-d49e-4888-a2c0-40a9e8af90d3] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:25:47.651426 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-7d37d71e-e57f-4dda-a4a1-d8b9b04e5dec tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:25:47.651909 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-7d37d71e-e57f-4dda-a4a1-d8b9b04e5dec tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:25:47.652334 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-7d37d71e-e57f-4dda-a4a1-d8b9b04e5dec tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:25:47.653240 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-7d37d71e-e57f-4dda-a4a1-d8b9b04e5dec tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:25:47.653689 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-7d37d71e-e57f-4dda-a4a1-d8b9b04e5dec tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:25:47.654109 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-7d37d71e-e57f-4dda-a4a1-d8b9b04e5dec tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:25:47.764433 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-7d37d71e-e57f-4dda-a4a1-d8b9b04e5dec tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd945d512ae034ec08838d0a3afde2429', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '880ac4003248441c809a76cfd1b221eb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:25:47.774932 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-7d37d71e-e57f-4dda-a4a1-d8b9b04e5dec tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] 10.222.0.52 "GET /compute/v2.1/servers/98e4cfeb-d49e-4888-a2c0-40a9e8af90d3" status: 200 len: 1296 microversion: 2.1 time: 0.146077 Aug 27 02:25:47.775606 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 223/443] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:25:47 2022] GET /compute/v2.1/servers/98e4cfeb-d49e-4888-a2c0-40a9e8af90d3 => generated 1296 bytes in 147 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:25:48.350854 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-4e10278f-ed84-4cb3-be28-c36ad68e036d tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] Calling method '>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:25:48.351453 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-4e10278f-ed84-4cb3-be28-c36ad68e036d tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] [instance: 9941c49f-8626-461d-a570-c0ffbfd4b804] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:25:48.357745 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-4e10278f-ed84-4cb3-be28-c36ad68e036d tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:25:48.358228 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-4e10278f-ed84-4cb3-be28-c36ad68e036d tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:25:48.358658 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-4e10278f-ed84-4cb3-be28-c36ad68e036d tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:25:48.359350 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-4e10278f-ed84-4cb3-be28-c36ad68e036d tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:25:48.359803 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-4e10278f-ed84-4cb3-be28-c36ad68e036d tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:25:48.360220 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-4e10278f-ed84-4cb3-be28-c36ad68e036d tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:25:48.366051 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG dbcounter [-] [120864] Writing DB stats nova_cell1:SELECT=99,nova_cell1:INSERT=1 {{(pid=120864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 27 02:25:48.494334 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-4e10278f-ed84-4cb3-be28-c36ad68e036d tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '642291c4daa04a6ab3fbce92a8be39c6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b2ddff853b7546788b645a0344c2297b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:25:48.507443 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-4e10278f-ed84-4cb3-be28-c36ad68e036d tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] 10.222.0.52 "GET /compute/v2.1/servers/9941c49f-8626-461d-a570-c0ffbfd4b804" status: 200 len: 1260 microversion: 2.1 time: 0.160955 Aug 27 02:25:48.508789 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 221/444] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:25:48 2022] GET /compute/v2.1/servers/9941c49f-8626-461d-a570-c0ffbfd4b804 => generated 1260 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:25:48.639553 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-3ec9847e-baea-4c96-af1d-f2dc4ef682f6 tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:25:48.640132 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-3ec9847e-baea-4c96-af1d-f2dc4ef682f6 tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] [instance: fbea9949-6c55-4ae7-96e1-b09f1a8030f1] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:25:48.646937 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-3ec9847e-baea-4c96-af1d-f2dc4ef682f6 tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:25:48.647431 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-3ec9847e-baea-4c96-af1d-f2dc4ef682f6 tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:25:48.647843 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-3ec9847e-baea-4c96-af1d-f2dc4ef682f6 tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:25:48.648292 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-5070eb4e-db95-4a47-864a-e413ebe4f462 tempest-VolumesSnapshotTestJSON-1750456804 tempest-VolumesSnapshotTestJSON-1750456804-project] Calling method '>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:25:48.648538 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-3ec9847e-baea-4c96-af1d-f2dc4ef682f6 tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:25:48.648884 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-5070eb4e-db95-4a47-864a-e413ebe4f462 tempest-VolumesSnapshotTestJSON-1750456804 tempest-VolumesSnapshotTestJSON-1750456804-project] [instance: c31c26a8-fe5a-40fb-91a1-7b5251fd0daa] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:25:48.649206 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-3ec9847e-baea-4c96-af1d-f2dc4ef682f6 tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:25:48.649633 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-3ec9847e-baea-4c96-af1d-f2dc4ef682f6 tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:25:48.656990 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-5070eb4e-db95-4a47-864a-e413ebe4f462 tempest-VolumesSnapshotTestJSON-1750456804 tempest-VolumesSnapshotTestJSON-1750456804-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:25:48.657734 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-5070eb4e-db95-4a47-864a-e413ebe4f462 tempest-VolumesSnapshotTestJSON-1750456804 tempest-VolumesSnapshotTestJSON-1750456804-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:25:48.658166 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-5070eb4e-db95-4a47-864a-e413ebe4f462 tempest-VolumesSnapshotTestJSON-1750456804 tempest-VolumesSnapshotTestJSON-1750456804-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:25:48.658870 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-5070eb4e-db95-4a47-864a-e413ebe4f462 tempest-VolumesSnapshotTestJSON-1750456804 tempest-VolumesSnapshotTestJSON-1750456804-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:25:48.659303 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-5070eb4e-db95-4a47-864a-e413ebe4f462 tempest-VolumesSnapshotTestJSON-1750456804 tempest-VolumesSnapshotTestJSON-1750456804-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:25:48.660124 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-5070eb4e-db95-4a47-864a-e413ebe4f462 tempest-VolumesSnapshotTestJSON-1750456804 tempest-VolumesSnapshotTestJSON-1750456804-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:25:48.857898 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-5070eb4e-db95-4a47-864a-e413ebe4f462 tempest-VolumesSnapshotTestJSON-1750456804 tempest-VolumesSnapshotTestJSON-1750456804-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '75744eccc3d24d4c9282cbe3a3314489', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fa43d54225bf44b283f9db5ddbbb13ca', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:25:48.864279 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-3ec9847e-baea-4c96-af1d-f2dc4ef682f6 tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2e60cf2931f04f0e8e911adea32979dd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7cb3e3496c1a45c8942c3678c0c299e5', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:25:48.869630 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-5070eb4e-db95-4a47-864a-e413ebe4f462 tempest-VolumesSnapshotTestJSON-1750456804 tempest-VolumesSnapshotTestJSON-1750456804-project] 10.222.0.52 "GET /compute/v2.1/servers/c31c26a8-fe5a-40fb-91a1-7b5251fd0daa" status: 200 len: 1319 microversion: 2.1 time: 0.225308 Aug 27 02:25:48.870858 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 222/445] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:25:48 2022] GET /compute/v2.1/servers/c31c26a8-fe5a-40fb-91a1-7b5251fd0daa => generated 1319 bytes in 227 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:25:48.875214 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-3ec9847e-baea-4c96-af1d-f2dc4ef682f6 tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] 10.222.0.52 "GET /compute/v2.1/servers/fbea9949-6c55-4ae7-96e1-b09f1a8030f1" status: 200 len: 1314 microversion: 2.1 time: 0.239513 Aug 27 02:25:48.875380 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [req-5884b690-de3f-4045-8a76-b475277427f9 req-139a5b06-686a-4427-9b6a-7efbe9e1dd4c service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "fbea9949-6c55-4ae7-96e1-b09f1a8030f1", "tag": "0d4a34a6-1648-44be-9eb9-79840124e8e3"}]} {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 27 02:25:48.885150 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 224/446] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:25:48 2022] GET /compute/v2.1/servers/fbea9949-6c55-4ae7-96e1-b09f1a8030f1 => generated 1314 bytes in 241 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:25:48.885150 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-7483a2e1-8254-415e-b6b2-a1b85ec36a07 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:25:48.885150 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-7483a2e1-8254-415e-b6b2-a1b85ec36a07 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] [instance: 98e4cfeb-d49e-4888-a2c0-40a9e8af90d3] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:25:48.885947 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [req-5884b690-de3f-4045-8a76-b475277427f9 req-139a5b06-686a-4427-9b6a-7efbe9e1dd4c service nova] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:25:48.886429 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [req-5884b690-de3f-4045-8a76-b475277427f9 req-139a5b06-686a-4427-9b6a-7efbe9e1dd4c service nova] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:25:48.886846 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [req-5884b690-de3f-4045-8a76-b475277427f9 req-139a5b06-686a-4427-9b6a-7efbe9e1dd4c service nova] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:25:48.891257 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-7483a2e1-8254-415e-b6b2-a1b85ec36a07 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:25:48.891743 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-7483a2e1-8254-415e-b6b2-a1b85ec36a07 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:25:48.892177 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-7483a2e1-8254-415e-b6b2-a1b85ec36a07 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:25:48.893048 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-7483a2e1-8254-415e-b6b2-a1b85ec36a07 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:25:48.893555 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-7483a2e1-8254-415e-b6b2-a1b85ec36a07 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:25:48.893980 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-7483a2e1-8254-415e-b6b2-a1b85ec36a07 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:25:48.898582 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.compute.server_external_events [req-5884b690-de3f-4045-8a76-b475277427f9 req-139a5b06-686a-4427-9b6a-7efbe9e1dd4c service nova] Creating event network-changed:0d4a34a6-1648-44be-9eb9-79840124e8e3 for instance fbea9949-6c55-4ae7-96e1-b09f1a8030f1 on hv-cinder-84630 Aug 27 02:25:48.903415 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [req-5884b690-de3f-4045-8a76-b475277427f9 req-139a5b06-686a-4427-9b6a-7efbe9e1dd4c service nova] 10.222.0.52 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.031609 Aug 27 02:25:48.904034 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 223/447] 10.222.0.52 () {64 vars in 1317 bytes} [Sat Aug 27 02:25:48 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 33 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 27 02:25:48.960005 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-7483a2e1-8254-415e-b6b2-a1b85ec36a07 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd945d512ae034ec08838d0a3afde2429', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '880ac4003248441c809a76cfd1b221eb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:25:48.976260 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-7483a2e1-8254-415e-b6b2-a1b85ec36a07 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] 10.222.0.52 "GET /compute/v2.1/servers/98e4cfeb-d49e-4888-a2c0-40a9e8af90d3" status: 200 len: 1314 microversion: 2.1 time: 0.098782 Aug 27 02:25:48.977058 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 225/448] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:25:48 2022] GET /compute/v2.1/servers/98e4cfeb-d49e-4888-a2c0-40a9e8af90d3 => generated 1314 bytes in 100 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:25:49.531815 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-0780336a-f7b3-48c0-805a-e8d87a9a24fc tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] Calling method '>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:25:49.532900 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-0780336a-f7b3-48c0-805a-e8d87a9a24fc tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] [instance: 9941c49f-8626-461d-a570-c0ffbfd4b804] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:25:49.540383 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-0780336a-f7b3-48c0-805a-e8d87a9a24fc tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:25:49.540869 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-0780336a-f7b3-48c0-805a-e8d87a9a24fc tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:25:49.541344 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-0780336a-f7b3-48c0-805a-e8d87a9a24fc tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:25:49.542046 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-0780336a-f7b3-48c0-805a-e8d87a9a24fc tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:25:49.542495 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-0780336a-f7b3-48c0-805a-e8d87a9a24fc tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:25:49.542917 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-0780336a-f7b3-48c0-805a-e8d87a9a24fc tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:25:49.694302 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-0780336a-f7b3-48c0-805a-e8d87a9a24fc tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '642291c4daa04a6ab3fbce92a8be39c6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b2ddff853b7546788b645a0344c2297b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:25:49.702469 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-0780336a-f7b3-48c0-805a-e8d87a9a24fc tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] 10.222.0.52 "GET /compute/v2.1/servers/9941c49f-8626-461d-a570-c0ffbfd4b804" status: 200 len: 1260 microversion: 2.1 time: 0.176815 Aug 27 02:25:49.703063 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 224/449] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:25:49 2022] GET /compute/v2.1/servers/9941c49f-8626-461d-a570-c0ffbfd4b804 => generated 1260 bytes in 178 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:25:49.897437 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-e8416ebb-d6a1-414e-885d-cfe78e4dfd6d tempest-VolumesSnapshotTestJSON-1750456804 tempest-VolumesSnapshotTestJSON-1750456804-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:25:49.898049 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-e8416ebb-d6a1-414e-885d-cfe78e4dfd6d tempest-VolumesSnapshotTestJSON-1750456804 tempest-VolumesSnapshotTestJSON-1750456804-project] [instance: c31c26a8-fe5a-40fb-91a1-7b5251fd0daa] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:25:49.900162 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-5360ea8b-1190-4944-846a-1a1f0f451246 tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Calling method '>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:25:49.900837 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-5360ea8b-1190-4944-846a-1a1f0f451246 tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] [instance: fbea9949-6c55-4ae7-96e1-b09f1a8030f1] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:25:49.904877 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-e8416ebb-d6a1-414e-885d-cfe78e4dfd6d tempest-VolumesSnapshotTestJSON-1750456804 tempest-VolumesSnapshotTestJSON-1750456804-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:25:49.905392 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-e8416ebb-d6a1-414e-885d-cfe78e4dfd6d tempest-VolumesSnapshotTestJSON-1750456804 tempest-VolumesSnapshotTestJSON-1750456804-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:25:49.905828 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-e8416ebb-d6a1-414e-885d-cfe78e4dfd6d tempest-VolumesSnapshotTestJSON-1750456804 tempest-VolumesSnapshotTestJSON-1750456804-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:25:49.906532 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-e8416ebb-d6a1-414e-885d-cfe78e4dfd6d tempest-VolumesSnapshotTestJSON-1750456804 tempest-VolumesSnapshotTestJSON-1750456804-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:25:49.906982 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-e8416ebb-d6a1-414e-885d-cfe78e4dfd6d tempest-VolumesSnapshotTestJSON-1750456804 tempest-VolumesSnapshotTestJSON-1750456804-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:25:49.907294 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-5360ea8b-1190-4944-846a-1a1f0f451246 tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:25:49.907396 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-e8416ebb-d6a1-414e-885d-cfe78e4dfd6d tempest-VolumesSnapshotTestJSON-1750456804 tempest-VolumesSnapshotTestJSON-1750456804-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:25:49.907776 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-5360ea8b-1190-4944-846a-1a1f0f451246 tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:25:49.908203 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-5360ea8b-1190-4944-846a-1a1f0f451246 tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:25:49.908895 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-5360ea8b-1190-4944-846a-1a1f0f451246 tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:25:49.909382 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-5360ea8b-1190-4944-846a-1a1f0f451246 tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:25:49.910119 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-5360ea8b-1190-4944-846a-1a1f0f451246 tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:25:50.098605 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-e8416ebb-d6a1-414e-885d-cfe78e4dfd6d tempest-VolumesSnapshotTestJSON-1750456804 tempest-VolumesSnapshotTestJSON-1750456804-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '75744eccc3d24d4c9282cbe3a3314489', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fa43d54225bf44b283f9db5ddbbb13ca', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:25:50.107119 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-e8416ebb-d6a1-414e-885d-cfe78e4dfd6d tempest-VolumesSnapshotTestJSON-1750456804 tempest-VolumesSnapshotTestJSON-1750456804-project] 10.222.0.52 "GET /compute/v2.1/servers/c31c26a8-fe5a-40fb-91a1-7b5251fd0daa" status: 200 len: 1319 microversion: 2.1 time: 0.214407 Aug 27 02:25:50.107506 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 226/450] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:25:49 2022] GET /compute/v2.1/servers/c31c26a8-fe5a-40fb-91a1-7b5251fd0daa => generated 1319 bytes in 216 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:25:50.109959 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-5360ea8b-1190-4944-846a-1a1f0f451246 tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2e60cf2931f04f0e8e911adea32979dd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7cb3e3496c1a45c8942c3678c0c299e5', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:25:50.112241 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-e11030a5-c2c0-4f48-9381-248f91bf6fff tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:25:50.112839 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-e11030a5-c2c0-4f48-9381-248f91bf6fff tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] [instance: 98e4cfeb-d49e-4888-a2c0-40a9e8af90d3] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:25:50.118680 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-5360ea8b-1190-4944-846a-1a1f0f451246 tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] 10.222.0.52 "GET /compute/v2.1/servers/fbea9949-6c55-4ae7-96e1-b09f1a8030f1" status: 200 len: 1314 microversion: 2.1 time: 0.222286 Aug 27 02:25:50.119444 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 225/451] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:25:49 2022] GET /compute/v2.1/servers/fbea9949-6c55-4ae7-96e1-b09f1a8030f1 => generated 1314 bytes in 223 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:25:50.121147 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-e11030a5-c2c0-4f48-9381-248f91bf6fff tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:25:50.121662 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-e11030a5-c2c0-4f48-9381-248f91bf6fff tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:25:50.122105 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-e11030a5-c2c0-4f48-9381-248f91bf6fff tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:25:50.122830 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-e11030a5-c2c0-4f48-9381-248f91bf6fff tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:25:50.123289 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-e11030a5-c2c0-4f48-9381-248f91bf6fff tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:25:50.123771 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-e11030a5-c2c0-4f48-9381-248f91bf6fff tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:25:50.187752 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-e11030a5-c2c0-4f48-9381-248f91bf6fff tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd945d512ae034ec08838d0a3afde2429', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '880ac4003248441c809a76cfd1b221eb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:25:50.195804 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-e11030a5-c2c0-4f48-9381-248f91bf6fff tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] 10.222.0.52 "GET /compute/v2.1/servers/98e4cfeb-d49e-4888-a2c0-40a9e8af90d3" status: 200 len: 1314 microversion: 2.1 time: 0.087329 Aug 27 02:25:50.196390 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 227/452] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:25:50 2022] GET /compute/v2.1/servers/98e4cfeb-d49e-4888-a2c0-40a9e8af90d3 => generated 1314 bytes in 88 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:25:50.727492 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-827bddaa-b0af-45da-b745-9412befda6f6 tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] Calling method '>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:25:50.728378 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-827bddaa-b0af-45da-b745-9412befda6f6 tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] [instance: 9941c49f-8626-461d-a570-c0ffbfd4b804] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:25:50.738146 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-827bddaa-b0af-45da-b745-9412befda6f6 tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:25:50.738948 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-827bddaa-b0af-45da-b745-9412befda6f6 tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:25:50.739408 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-827bddaa-b0af-45da-b745-9412befda6f6 tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:25:50.740088 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-827bddaa-b0af-45da-b745-9412befda6f6 tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:25:50.740564 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-827bddaa-b0af-45da-b745-9412befda6f6 tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:25:50.740977 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-827bddaa-b0af-45da-b745-9412befda6f6 tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:25:50.887245 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-827bddaa-b0af-45da-b745-9412befda6f6 tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '642291c4daa04a6ab3fbce92a8be39c6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b2ddff853b7546788b645a0344c2297b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:25:50.895851 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-827bddaa-b0af-45da-b745-9412befda6f6 tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] 10.222.0.52 "GET /compute/v2.1/servers/9941c49f-8626-461d-a570-c0ffbfd4b804" status: 200 len: 1260 microversion: 2.1 time: 0.175300 Aug 27 02:25:50.896427 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 226/453] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:25:50 2022] GET /compute/v2.1/servers/9941c49f-8626-461d-a570-c0ffbfd4b804 => generated 1260 bytes in 177 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:25:51.128462 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-13b6528c-4774-4c16-8335-9509dd4bfe96 tempest-VolumesSnapshotTestJSON-1750456804 tempest-VolumesSnapshotTestJSON-1750456804-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:25:51.129103 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-13b6528c-4774-4c16-8335-9509dd4bfe96 tempest-VolumesSnapshotTestJSON-1750456804 tempest-VolumesSnapshotTestJSON-1750456804-project] [instance: c31c26a8-fe5a-40fb-91a1-7b5251fd0daa] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:25:51.136023 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-13b6528c-4774-4c16-8335-9509dd4bfe96 tempest-VolumesSnapshotTestJSON-1750456804 tempest-VolumesSnapshotTestJSON-1750456804-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:25:51.136509 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-13b6528c-4774-4c16-8335-9509dd4bfe96 tempest-VolumesSnapshotTestJSON-1750456804 tempest-VolumesSnapshotTestJSON-1750456804-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:25:51.136949 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-13b6528c-4774-4c16-8335-9509dd4bfe96 tempest-VolumesSnapshotTestJSON-1750456804 tempest-VolumesSnapshotTestJSON-1750456804-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:25:51.137696 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-13b6528c-4774-4c16-8335-9509dd4bfe96 tempest-VolumesSnapshotTestJSON-1750456804 tempest-VolumesSnapshotTestJSON-1750456804-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:25:51.138146 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-13b6528c-4774-4c16-8335-9509dd4bfe96 tempest-VolumesSnapshotTestJSON-1750456804 tempest-VolumesSnapshotTestJSON-1750456804-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:25:51.138574 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-13b6528c-4774-4c16-8335-9509dd4bfe96 tempest-VolumesSnapshotTestJSON-1750456804 tempest-VolumesSnapshotTestJSON-1750456804-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:25:51.140012 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-f87bac65-61ef-4fe0-9568-69f1fa799e8e tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Calling method '>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:25:51.140646 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-f87bac65-61ef-4fe0-9568-69f1fa799e8e tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] [instance: fbea9949-6c55-4ae7-96e1-b09f1a8030f1] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:25:51.147139 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-f87bac65-61ef-4fe0-9568-69f1fa799e8e tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:25:51.147622 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-f87bac65-61ef-4fe0-9568-69f1fa799e8e tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:25:51.148034 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-f87bac65-61ef-4fe0-9568-69f1fa799e8e tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:25:51.148742 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-f87bac65-61ef-4fe0-9568-69f1fa799e8e tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:25:51.149285 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-f87bac65-61ef-4fe0-9568-69f1fa799e8e tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:25:51.149702 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-f87bac65-61ef-4fe0-9568-69f1fa799e8e tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:25:51.338939 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-13b6528c-4774-4c16-8335-9509dd4bfe96 tempest-VolumesSnapshotTestJSON-1750456804 tempest-VolumesSnapshotTestJSON-1750456804-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '75744eccc3d24d4c9282cbe3a3314489', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fa43d54225bf44b283f9db5ddbbb13ca', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:25:51.346903 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-13b6528c-4774-4c16-8335-9509dd4bfe96 tempest-VolumesSnapshotTestJSON-1750456804 tempest-VolumesSnapshotTestJSON-1750456804-project] 10.222.0.52 "GET /compute/v2.1/servers/c31c26a8-fe5a-40fb-91a1-7b5251fd0daa" status: 200 len: 1319 microversion: 2.1 time: 0.222556 Aug 27 02:25:51.356089 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 228/454] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:25:51 2022] GET /compute/v2.1/servers/c31c26a8-fe5a-40fb-91a1-7b5251fd0daa => generated 1319 bytes in 224 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:25:51.356371 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-f87bac65-61ef-4fe0-9568-69f1fa799e8e tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2e60cf2931f04f0e8e911adea32979dd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7cb3e3496c1a45c8942c3678c0c299e5', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:25:51.356529 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-fb30035a-c47b-479f-a289-de7d3f5ec78f tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:25:51.356529 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-fb30035a-c47b-479f-a289-de7d3f5ec78f tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] [instance: 98e4cfeb-d49e-4888-a2c0-40a9e8af90d3] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:25:51.356529 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG dbcounter [-] [120863] Writing DB stats nova_api:SELECT=97,nova_api:INSERT=3 {{(pid=120863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 27 02:25:51.364901 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-fb30035a-c47b-479f-a289-de7d3f5ec78f tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:25:51.365497 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-fb30035a-c47b-479f-a289-de7d3f5ec78f tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:25:51.366021 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-fb30035a-c47b-479f-a289-de7d3f5ec78f tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:25:51.366798 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-fb30035a-c47b-479f-a289-de7d3f5ec78f tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:25:51.367312 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-fb30035a-c47b-479f-a289-de7d3f5ec78f tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:25:51.367998 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-fb30035a-c47b-479f-a289-de7d3f5ec78f tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:25:51.384262 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-f87bac65-61ef-4fe0-9568-69f1fa799e8e tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] 10.222.0.52 "GET /compute/v2.1/servers/fbea9949-6c55-4ae7-96e1-b09f1a8030f1" status: 200 len: 1314 microversion: 2.1 time: 0.247995 Aug 27 02:25:51.384975 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 227/455] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:25:51 2022] GET /compute/v2.1/servers/fbea9949-6c55-4ae7-96e1-b09f1a8030f1 => generated 1314 bytes in 249 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 27 02:25:51.460714 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-fb30035a-c47b-479f-a289-de7d3f5ec78f tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd945d512ae034ec08838d0a3afde2429', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '880ac4003248441c809a76cfd1b221eb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:25:51.468521 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-fb30035a-c47b-479f-a289-de7d3f5ec78f tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] 10.222.0.52 "GET /compute/v2.1/servers/98e4cfeb-d49e-4888-a2c0-40a9e8af90d3" status: 200 len: 1314 microversion: 2.1 time: 0.119633 Aug 27 02:25:51.469334 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 229/456] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:25:51 2022] GET /compute/v2.1/servers/98e4cfeb-d49e-4888-a2c0-40a9e8af90d3 => generated 1314 bytes in 121 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:25:51.918981 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-e77557e0-c182-4582-9d03-c510fdb17c26 tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] Calling method '>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:25:51.920127 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-e77557e0-c182-4582-9d03-c510fdb17c26 tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] [instance: 9941c49f-8626-461d-a570-c0ffbfd4b804] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:25:51.928815 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-e77557e0-c182-4582-9d03-c510fdb17c26 tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:25:51.930369 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-e77557e0-c182-4582-9d03-c510fdb17c26 tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:25:51.930369 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-e77557e0-c182-4582-9d03-c510fdb17c26 tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:25:51.931262 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-e77557e0-c182-4582-9d03-c510fdb17c26 tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:25:51.931262 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-e77557e0-c182-4582-9d03-c510fdb17c26 tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:25:51.931262 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-e77557e0-c182-4582-9d03-c510fdb17c26 tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:25:52.077311 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-e77557e0-c182-4582-9d03-c510fdb17c26 tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '642291c4daa04a6ab3fbce92a8be39c6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b2ddff853b7546788b645a0344c2297b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:25:52.085480 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-e77557e0-c182-4582-9d03-c510fdb17c26 tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] 10.222.0.52 "GET /compute/v2.1/servers/9941c49f-8626-461d-a570-c0ffbfd4b804" status: 200 len: 1260 microversion: 2.1 time: 0.172545 Aug 27 02:25:52.086093 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 228/457] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:25:51 2022] GET /compute/v2.1/servers/9941c49f-8626-461d-a570-c0ffbfd4b804 => generated 1260 bytes in 174 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 27 02:25:52.373406 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-d6f68cac-c749-4c7d-9f62-90ba7fd3b4ad tempest-VolumesSnapshotTestJSON-1750456804 tempest-VolumesSnapshotTestJSON-1750456804-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:25:52.374033 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-d6f68cac-c749-4c7d-9f62-90ba7fd3b4ad tempest-VolumesSnapshotTestJSON-1750456804 tempest-VolumesSnapshotTestJSON-1750456804-project] [instance: c31c26a8-fe5a-40fb-91a1-7b5251fd0daa] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:25:52.382178 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-d6f68cac-c749-4c7d-9f62-90ba7fd3b4ad tempest-VolumesSnapshotTestJSON-1750456804 tempest-VolumesSnapshotTestJSON-1750456804-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:25:52.382649 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-d6f68cac-c749-4c7d-9f62-90ba7fd3b4ad tempest-VolumesSnapshotTestJSON-1750456804 tempest-VolumesSnapshotTestJSON-1750456804-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:25:52.383080 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-d6f68cac-c749-4c7d-9f62-90ba7fd3b4ad tempest-VolumesSnapshotTestJSON-1750456804 tempest-VolumesSnapshotTestJSON-1750456804-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:25:52.383771 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-d6f68cac-c749-4c7d-9f62-90ba7fd3b4ad tempest-VolumesSnapshotTestJSON-1750456804 tempest-VolumesSnapshotTestJSON-1750456804-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:25:52.384223 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-d6f68cac-c749-4c7d-9f62-90ba7fd3b4ad tempest-VolumesSnapshotTestJSON-1750456804 tempest-VolumesSnapshotTestJSON-1750456804-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:25:52.384641 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-d6f68cac-c749-4c7d-9f62-90ba7fd3b4ad tempest-VolumesSnapshotTestJSON-1750456804 tempest-VolumesSnapshotTestJSON-1750456804-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:25:52.407434 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-d49068ae-29a2-469e-baa0-9b91b170b310 tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Calling method '>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:25:52.408034 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-d49068ae-29a2-469e-baa0-9b91b170b310 tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] [instance: fbea9949-6c55-4ae7-96e1-b09f1a8030f1] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:25:52.414977 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-d49068ae-29a2-469e-baa0-9b91b170b310 tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:25:52.415492 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-d49068ae-29a2-469e-baa0-9b91b170b310 tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:25:52.415916 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-d49068ae-29a2-469e-baa0-9b91b170b310 tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:25:52.416609 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-d49068ae-29a2-469e-baa0-9b91b170b310 tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:25:52.417113 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-d49068ae-29a2-469e-baa0-9b91b170b310 tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:25:52.417579 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-d49068ae-29a2-469e-baa0-9b91b170b310 tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:25:52.595326 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-d6f68cac-c749-4c7d-9f62-90ba7fd3b4ad tempest-VolumesSnapshotTestJSON-1750456804 tempest-VolumesSnapshotTestJSON-1750456804-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '75744eccc3d24d4c9282cbe3a3314489', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fa43d54225bf44b283f9db5ddbbb13ca', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:25:52.603178 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-d49068ae-29a2-469e-baa0-9b91b170b310 tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2e60cf2931f04f0e8e911adea32979dd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7cb3e3496c1a45c8942c3678c0c299e5', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:25:52.603980 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-d6f68cac-c749-4c7d-9f62-90ba7fd3b4ad tempest-VolumesSnapshotTestJSON-1750456804 tempest-VolumesSnapshotTestJSON-1750456804-project] 10.222.0.52 "GET /compute/v2.1/servers/c31c26a8-fe5a-40fb-91a1-7b5251fd0daa" status: 200 len: 1319 microversion: 2.1 time: 0.236601 Aug 27 02:25:52.604629 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 230/458] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:25:52 2022] GET /compute/v2.1/servers/c31c26a8-fe5a-40fb-91a1-7b5251fd0daa => generated 1319 bytes in 238 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:25:52.609630 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-6b5562b9-246d-4c32-a423-7280ccd36b50 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:25:52.610249 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-6b5562b9-246d-4c32-a423-7280ccd36b50 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] [instance: 98e4cfeb-d49e-4888-a2c0-40a9e8af90d3] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:25:52.611785 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-d49068ae-29a2-469e-baa0-9b91b170b310 tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] 10.222.0.52 "GET /compute/v2.1/servers/fbea9949-6c55-4ae7-96e1-b09f1a8030f1" status: 200 len: 1314 microversion: 2.1 time: 0.207978 Aug 27 02:25:52.612360 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 229/459] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:25:52 2022] GET /compute/v2.1/servers/fbea9949-6c55-4ae7-96e1-b09f1a8030f1 => generated 1314 bytes in 209 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:25:52.616546 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-6b5562b9-246d-4c32-a423-7280ccd36b50 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:25:52.617038 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-6b5562b9-246d-4c32-a423-7280ccd36b50 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:25:52.617507 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-6b5562b9-246d-4c32-a423-7280ccd36b50 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:25:52.618244 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-6b5562b9-246d-4c32-a423-7280ccd36b50 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:25:52.618705 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-6b5562b9-246d-4c32-a423-7280ccd36b50 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:25:52.619164 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-6b5562b9-246d-4c32-a423-7280ccd36b50 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:25:52.690713 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-6b5562b9-246d-4c32-a423-7280ccd36b50 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd945d512ae034ec08838d0a3afde2429', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '880ac4003248441c809a76cfd1b221eb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:25:52.699007 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-6b5562b9-246d-4c32-a423-7280ccd36b50 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] 10.222.0.52 "GET /compute/v2.1/servers/98e4cfeb-d49e-4888-a2c0-40a9e8af90d3" status: 200 len: 1302 microversion: 2.1 time: 0.093147 Aug 27 02:25:52.699849 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 231/460] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:25:52 2022] GET /compute/v2.1/servers/98e4cfeb-d49e-4888-a2c0-40a9e8af90d3 => generated 1302 bytes in 94 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:25:53.109197 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-de7ba0ab-9860-4567-af38-d67811a2c761 tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] Calling method '>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:25:53.110175 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-de7ba0ab-9860-4567-af38-d67811a2c761 tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] [instance: 9941c49f-8626-461d-a570-c0ffbfd4b804] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:25:53.118906 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-de7ba0ab-9860-4567-af38-d67811a2c761 tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:25:53.119524 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-de7ba0ab-9860-4567-af38-d67811a2c761 tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:25:53.120043 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-de7ba0ab-9860-4567-af38-d67811a2c761 tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:25:53.120989 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-de7ba0ab-9860-4567-af38-d67811a2c761 tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:25:53.121801 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-de7ba0ab-9860-4567-af38-d67811a2c761 tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:25:53.122314 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-de7ba0ab-9860-4567-af38-d67811a2c761 tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:25:53.439216 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-de7ba0ab-9860-4567-af38-d67811a2c761 tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '642291c4daa04a6ab3fbce92a8be39c6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b2ddff853b7546788b645a0344c2297b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:25:53.447563 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-de7ba0ab-9860-4567-af38-d67811a2c761 tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] 10.222.0.52 "GET /compute/v2.1/servers/9941c49f-8626-461d-a570-c0ffbfd4b804" status: 200 len: 1260 microversion: 2.1 time: 0.344487 Aug 27 02:25:53.448197 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 230/461] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:25:53 2022] GET /compute/v2.1/servers/9941c49f-8626-461d-a570-c0ffbfd4b804 => generated 1260 bytes in 346 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:25:53.626524 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-f68571f3-8431-445e-8a51-940217cea368 tempest-VolumesSnapshotTestJSON-1750456804 tempest-VolumesSnapshotTestJSON-1750456804-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:25:53.626882 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-f68571f3-8431-445e-8a51-940217cea368 tempest-VolumesSnapshotTestJSON-1750456804 tempest-VolumesSnapshotTestJSON-1750456804-project] [instance: c31c26a8-fe5a-40fb-91a1-7b5251fd0daa] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:25:53.633676 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-9ab29527-93b2-4d55-9f8c-cdb5358b8d6c tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Calling method '>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:25:53.634283 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-9ab29527-93b2-4d55-9f8c-cdb5358b8d6c tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] [instance: fbea9949-6c55-4ae7-96e1-b09f1a8030f1] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:25:53.635328 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-f68571f3-8431-445e-8a51-940217cea368 tempest-VolumesSnapshotTestJSON-1750456804 tempest-VolumesSnapshotTestJSON-1750456804-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:25:53.636364 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-f68571f3-8431-445e-8a51-940217cea368 tempest-VolumesSnapshotTestJSON-1750456804 tempest-VolumesSnapshotTestJSON-1750456804-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:25:53.637724 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-f68571f3-8431-445e-8a51-940217cea368 tempest-VolumesSnapshotTestJSON-1750456804 tempest-VolumesSnapshotTestJSON-1750456804-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:25:53.638769 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-f68571f3-8431-445e-8a51-940217cea368 tempest-VolumesSnapshotTestJSON-1750456804 tempest-VolumesSnapshotTestJSON-1750456804-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:25:53.639677 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-f68571f3-8431-445e-8a51-940217cea368 tempest-VolumesSnapshotTestJSON-1750456804 tempest-VolumesSnapshotTestJSON-1750456804-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:25:53.640338 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-f68571f3-8431-445e-8a51-940217cea368 tempest-VolumesSnapshotTestJSON-1750456804 tempest-VolumesSnapshotTestJSON-1750456804-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:25:53.640579 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-9ab29527-93b2-4d55-9f8c-cdb5358b8d6c tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:25:53.641271 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-9ab29527-93b2-4d55-9f8c-cdb5358b8d6c tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:25:53.641877 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-9ab29527-93b2-4d55-9f8c-cdb5358b8d6c tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:25:53.642732 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-9ab29527-93b2-4d55-9f8c-cdb5358b8d6c tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:25:53.643337 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-9ab29527-93b2-4d55-9f8c-cdb5358b8d6c tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:25:53.644062 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-9ab29527-93b2-4d55-9f8c-cdb5358b8d6c tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:25:54.260001 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-f68571f3-8431-445e-8a51-940217cea368 tempest-VolumesSnapshotTestJSON-1750456804 tempest-VolumesSnapshotTestJSON-1750456804-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '75744eccc3d24d4c9282cbe3a3314489', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fa43d54225bf44b283f9db5ddbbb13ca', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:25:54.269654 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-f68571f3-8431-445e-8a51-940217cea368 tempest-VolumesSnapshotTestJSON-1750456804 tempest-VolumesSnapshotTestJSON-1750456804-project] 10.222.0.52 "GET /compute/v2.1/servers/c31c26a8-fe5a-40fb-91a1-7b5251fd0daa" status: 200 len: 1319 microversion: 2.1 time: 0.647661 Aug 27 02:25:54.270306 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 232/462] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:25:53 2022] GET /compute/v2.1/servers/c31c26a8-fe5a-40fb-91a1-7b5251fd0daa => generated 1319 bytes in 649 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:25:54.275089 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-18b59b22-5fef-43ff-af52-16382e2ef11c tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:25:54.276262 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-18b59b22-5fef-43ff-af52-16382e2ef11c tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] [instance: 98e4cfeb-d49e-4888-a2c0-40a9e8af90d3] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:25:54.279216 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-9ab29527-93b2-4d55-9f8c-cdb5358b8d6c tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2e60cf2931f04f0e8e911adea32979dd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7cb3e3496c1a45c8942c3678c0c299e5', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:25:54.283121 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-18b59b22-5fef-43ff-af52-16382e2ef11c tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:25:54.284004 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-18b59b22-5fef-43ff-af52-16382e2ef11c tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:25:54.284004 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-18b59b22-5fef-43ff-af52-16382e2ef11c tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:25:54.284702 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-18b59b22-5fef-43ff-af52-16382e2ef11c tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:25:54.285255 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-18b59b22-5fef-43ff-af52-16382e2ef11c tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:25:54.285967 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-18b59b22-5fef-43ff-af52-16382e2ef11c tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:25:54.292342 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-9ab29527-93b2-4d55-9f8c-cdb5358b8d6c tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] 10.222.0.52 "GET /compute/v2.1/servers/fbea9949-6c55-4ae7-96e1-b09f1a8030f1" status: 200 len: 1314 microversion: 2.1 time: 0.662364 Aug 27 02:25:54.293459 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 231/463] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:25:53 2022] GET /compute/v2.1/servers/fbea9949-6c55-4ae7-96e1-b09f1a8030f1 => generated 1314 bytes in 664 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 27 02:25:54.299514 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [req-9bee81f8-d022-4e43-b12f-412bb19301e4 req-4bc819de-34dc-4a66-89e4-3ec1fcaa0358 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "c31c26a8-fe5a-40fb-91a1-7b5251fd0daa", "name": "network-vif-plugged", "status": "completed", "tag": "22831215-7029-41bf-a558-6aa1056c1015"}]} {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 27 02:25:54.311419 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [req-9bee81f8-d022-4e43-b12f-412bb19301e4 req-4bc819de-34dc-4a66-89e4-3ec1fcaa0358 service nova] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:25:54.312267 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [req-9bee81f8-d022-4e43-b12f-412bb19301e4 req-4bc819de-34dc-4a66-89e4-3ec1fcaa0358 service nova] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:25:54.312995 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [req-9bee81f8-d022-4e43-b12f-412bb19301e4 req-4bc819de-34dc-4a66-89e4-3ec1fcaa0358 service nova] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:25:54.328164 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.compute.server_external_events [req-9bee81f8-d022-4e43-b12f-412bb19301e4 req-4bc819de-34dc-4a66-89e4-3ec1fcaa0358 service nova] Creating event network-vif-plugged:22831215-7029-41bf-a558-6aa1056c1015 for instance c31c26a8-fe5a-40fb-91a1-7b5251fd0daa on hv-cinder-84630 Aug 27 02:25:54.334039 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [req-9bee81f8-d022-4e43-b12f-412bb19301e4 req-4bc819de-34dc-4a66-89e4-3ec1fcaa0358 service nova] 10.222.0.52 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.039239 Aug 27 02:25:54.334688 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 232/464] 10.222.0.52 () {64 vars in 1317 bytes} [Sat Aug 27 02:25:54 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 41 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 27 02:25:54.471060 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-387f66e3-1f25-4551-bc3d-b338494b06d6 tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] Calling method '>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:25:54.471754 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-387f66e3-1f25-4551-bc3d-b338494b06d6 tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] [instance: 9941c49f-8626-461d-a570-c0ffbfd4b804] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:25:54.479752 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-387f66e3-1f25-4551-bc3d-b338494b06d6 tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:25:54.480686 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-387f66e3-1f25-4551-bc3d-b338494b06d6 tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:25:54.481385 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-387f66e3-1f25-4551-bc3d-b338494b06d6 tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:25:54.482209 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-387f66e3-1f25-4551-bc3d-b338494b06d6 tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:25:54.482663 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-387f66e3-1f25-4551-bc3d-b338494b06d6 tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:25:54.483075 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-387f66e3-1f25-4551-bc3d-b338494b06d6 tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:25:54.523686 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG dbcounter [-] [120863] Writing DB stats nova_cell0:SELECT=2 {{(pid=120863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 27 02:25:54.630520 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-18b59b22-5fef-43ff-af52-16382e2ef11c tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd945d512ae034ec08838d0a3afde2429', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '880ac4003248441c809a76cfd1b221eb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:25:54.639576 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-18b59b22-5fef-43ff-af52-16382e2ef11c tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] 10.222.0.52 "GET /compute/v2.1/servers/98e4cfeb-d49e-4888-a2c0-40a9e8af90d3" status: 200 len: 1370 microversion: 2.1 time: 0.367994 Aug 27 02:25:54.640871 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 233/465] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:25:54 2022] GET /compute/v2.1/servers/98e4cfeb-d49e-4888-a2c0-40a9e8af90d3 => generated 1370 bytes in 370 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:25:54.745587 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-387f66e3-1f25-4551-bc3d-b338494b06d6 tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '642291c4daa04a6ab3fbce92a8be39c6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b2ddff853b7546788b645a0344c2297b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:25:54.753426 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-387f66e3-1f25-4551-bc3d-b338494b06d6 tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] 10.222.0.52 "GET /compute/v2.1/servers/9941c49f-8626-461d-a570-c0ffbfd4b804" status: 200 len: 1260 microversion: 2.1 time: 0.287352 Aug 27 02:25:54.754158 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 233/466] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:25:54 2022] GET /compute/v2.1/servers/9941c49f-8626-461d-a570-c0ffbfd4b804 => generated 1260 bytes in 289 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 27 02:25:55.278635 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [req-eb789f4c-ef04-42bf-ac0c-6b55322348e5 req-8309ad35-aba8-4f4b-a84f-70eaa8d3614d service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "98e4cfeb-d49e-4888-a2c0-40a9e8af90d3", "tag": "bf498aa0-7dfa-466b-b4f0-3a5d8f8703e2"}]} {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 27 02:25:55.289069 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [req-eb789f4c-ef04-42bf-ac0c-6b55322348e5 req-8309ad35-aba8-4f4b-a84f-70eaa8d3614d service nova] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:25:55.289766 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [req-eb789f4c-ef04-42bf-ac0c-6b55322348e5 req-8309ad35-aba8-4f4b-a84f-70eaa8d3614d service nova] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:25:55.289981 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [req-eb789f4c-ef04-42bf-ac0c-6b55322348e5 req-8309ad35-aba8-4f4b-a84f-70eaa8d3614d service nova] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:25:55.292460 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-9691cb70-e990-4a49-8bb2-c6ecc9ac586d tempest-VolumesSnapshotTestJSON-1750456804 tempest-VolumesSnapshotTestJSON-1750456804-project] Calling method '>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:25:55.293780 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-9691cb70-e990-4a49-8bb2-c6ecc9ac586d tempest-VolumesSnapshotTestJSON-1750456804 tempest-VolumesSnapshotTestJSON-1750456804-project] [instance: c31c26a8-fe5a-40fb-91a1-7b5251fd0daa] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:25:55.302135 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-9691cb70-e990-4a49-8bb2-c6ecc9ac586d tempest-VolumesSnapshotTestJSON-1750456804 tempest-VolumesSnapshotTestJSON-1750456804-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:25:55.302634 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-9691cb70-e990-4a49-8bb2-c6ecc9ac586d tempest-VolumesSnapshotTestJSON-1750456804 tempest-VolumesSnapshotTestJSON-1750456804-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:25:55.302778 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.compute.server_external_events [req-eb789f4c-ef04-42bf-ac0c-6b55322348e5 req-8309ad35-aba8-4f4b-a84f-70eaa8d3614d service nova] Creating event network-changed:bf498aa0-7dfa-466b-b4f0-3a5d8f8703e2 for instance 98e4cfeb-d49e-4888-a2c0-40a9e8af90d3 on hv-cinder-84630 Aug 27 02:25:55.303065 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-9691cb70-e990-4a49-8bb2-c6ecc9ac586d tempest-VolumesSnapshotTestJSON-1750456804 tempest-VolumesSnapshotTestJSON-1750456804-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:25:55.304536 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-9691cb70-e990-4a49-8bb2-c6ecc9ac586d tempest-VolumesSnapshotTestJSON-1750456804 tempest-VolumesSnapshotTestJSON-1750456804-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:25:55.305844 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-9691cb70-e990-4a49-8bb2-c6ecc9ac586d tempest-VolumesSnapshotTestJSON-1750456804 tempest-VolumesSnapshotTestJSON-1750456804-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:25:55.308457 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-9691cb70-e990-4a49-8bb2-c6ecc9ac586d tempest-VolumesSnapshotTestJSON-1750456804 tempest-VolumesSnapshotTestJSON-1750456804-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:25:55.308661 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [req-eb789f4c-ef04-42bf-ac0c-6b55322348e5 req-8309ad35-aba8-4f4b-a84f-70eaa8d3614d service nova] 10.222.0.52 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.033341 Aug 27 02:25:55.309041 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 234/467] 10.222.0.52 () {64 vars in 1317 bytes} [Sat Aug 27 02:25:55 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 35 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 27 02:25:55.314338 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-bdd7cfed-52a8-4023-8842-32e755f078a2 tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:25:55.315080 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-bdd7cfed-52a8-4023-8842-32e755f078a2 tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] [instance: fbea9949-6c55-4ae7-96e1-b09f1a8030f1] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:25:55.321763 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-bdd7cfed-52a8-4023-8842-32e755f078a2 tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:25:55.322262 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-bdd7cfed-52a8-4023-8842-32e755f078a2 tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:25:55.322695 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-bdd7cfed-52a8-4023-8842-32e755f078a2 tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:25:55.323592 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-bdd7cfed-52a8-4023-8842-32e755f078a2 tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:25:55.324223 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-bdd7cfed-52a8-4023-8842-32e755f078a2 tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:25:55.325404 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-bdd7cfed-52a8-4023-8842-32e755f078a2 tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:25:55.533923 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-9691cb70-e990-4a49-8bb2-c6ecc9ac586d tempest-VolumesSnapshotTestJSON-1750456804 tempest-VolumesSnapshotTestJSON-1750456804-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '75744eccc3d24d4c9282cbe3a3314489', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fa43d54225bf44b283f9db5ddbbb13ca', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:25:55.543136 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-9691cb70-e990-4a49-8bb2-c6ecc9ac586d tempest-VolumesSnapshotTestJSON-1750456804 tempest-VolumesSnapshotTestJSON-1750456804-project] 10.222.0.52 "GET /compute/v2.1/servers/c31c26a8-fe5a-40fb-91a1-7b5251fd0daa" status: 200 len: 1319 microversion: 2.1 time: 0.255963 Aug 27 02:25:55.543306 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-bdd7cfed-52a8-4023-8842-32e755f078a2 tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2e60cf2931f04f0e8e911adea32979dd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7cb3e3496c1a45c8942c3678c0c299e5', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:25:55.543702 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 234/468] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:25:55 2022] GET /compute/v2.1/servers/c31c26a8-fe5a-40fb-91a1-7b5251fd0daa => generated 1319 bytes in 257 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:25:55.553871 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-bdd7cfed-52a8-4023-8842-32e755f078a2 tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] 10.222.0.52 "GET /compute/v2.1/servers/fbea9949-6c55-4ae7-96e1-b09f1a8030f1" status: 200 len: 1314 microversion: 2.1 time: 0.243520 Aug 27 02:25:55.554424 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 235/469] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:25:55 2022] GET /compute/v2.1/servers/fbea9949-6c55-4ae7-96e1-b09f1a8030f1 => generated 1314 bytes in 245 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:25:55.664643 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-38e8cfe2-4452-4829-9fd9-f03ee99fd2c2 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Calling method '>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:25:55.665517 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-38e8cfe2-4452-4829-9fd9-f03ee99fd2c2 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] [instance: 98e4cfeb-d49e-4888-a2c0-40a9e8af90d3] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:25:55.674098 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-38e8cfe2-4452-4829-9fd9-f03ee99fd2c2 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:25:55.674591 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-38e8cfe2-4452-4829-9fd9-f03ee99fd2c2 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:25:55.675207 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-38e8cfe2-4452-4829-9fd9-f03ee99fd2c2 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:25:55.677721 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-38e8cfe2-4452-4829-9fd9-f03ee99fd2c2 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:25:55.678215 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-38e8cfe2-4452-4829-9fd9-f03ee99fd2c2 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:25:55.678851 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-38e8cfe2-4452-4829-9fd9-f03ee99fd2c2 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:25:55.775609 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-8b754ca2-55e8-4d6e-b85b-04315ae1bbb5 tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:25:55.776161 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-8b754ca2-55e8-4d6e-b85b-04315ae1bbb5 tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] [instance: 9941c49f-8626-461d-a570-c0ffbfd4b804] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:25:55.785162 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-8b754ca2-55e8-4d6e-b85b-04315ae1bbb5 tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:25:55.785975 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-8b754ca2-55e8-4d6e-b85b-04315ae1bbb5 tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:25:55.786711 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-8b754ca2-55e8-4d6e-b85b-04315ae1bbb5 tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:25:55.787849 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-8b754ca2-55e8-4d6e-b85b-04315ae1bbb5 tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:25:55.788618 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-8b754ca2-55e8-4d6e-b85b-04315ae1bbb5 tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:25:55.789350 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-8b754ca2-55e8-4d6e-b85b-04315ae1bbb5 tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:25:55.845628 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-38e8cfe2-4452-4829-9fd9-f03ee99fd2c2 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd945d512ae034ec08838d0a3afde2429', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '880ac4003248441c809a76cfd1b221eb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:25:55.854243 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-38e8cfe2-4452-4829-9fd9-f03ee99fd2c2 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] 10.222.0.52 "GET /compute/v2.1/servers/98e4cfeb-d49e-4888-a2c0-40a9e8af90d3" status: 200 len: 1370 microversion: 2.1 time: 0.196288 Aug 27 02:25:55.854894 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 235/470] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:25:55 2022] GET /compute/v2.1/servers/98e4cfeb-d49e-4888-a2c0-40a9e8af90d3 => generated 1370 bytes in 198 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:25:55.943784 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-8b754ca2-55e8-4d6e-b85b-04315ae1bbb5 tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '642291c4daa04a6ab3fbce92a8be39c6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b2ddff853b7546788b645a0344c2297b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:25:55.954775 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-8b754ca2-55e8-4d6e-b85b-04315ae1bbb5 tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] 10.222.0.52 "GET /compute/v2.1/servers/9941c49f-8626-461d-a570-c0ffbfd4b804" status: 200 len: 1260 microversion: 2.1 time: 0.182885 Aug 27 02:25:55.955513 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 236/471] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:25:55 2022] GET /compute/v2.1/servers/9941c49f-8626-461d-a570-c0ffbfd4b804 => generated 1260 bytes in 185 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 27 02:25:56.365423 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [req-a52e134a-5c7a-4389-85d0-80f21789b79b req-5553c8e1-b9e4-42b7-b2b9-a8cc157b315f service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "9941c49f-8626-461d-a570-c0ffbfd4b804", "name": "network-vif-plugged", "status": "completed", "tag": "89f4d982-1807-4e80-b9ef-c7bc0025f1b0"}]} {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 27 02:25:56.370878 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG dbcounter [-] [120864] Writing DB stats nova_cell0:SELECT=6 {{(pid=120864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 27 02:25:56.371690 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG dbcounter [-] [120864] Writing DB stats nova_cell0:SELECT=22 {{(pid=120864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 27 02:25:56.378232 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [req-a52e134a-5c7a-4389-85d0-80f21789b79b req-5553c8e1-b9e4-42b7-b2b9-a8cc157b315f service nova] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:25:56.379165 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [req-a52e134a-5c7a-4389-85d0-80f21789b79b req-5553c8e1-b9e4-42b7-b2b9-a8cc157b315f service nova] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:25:56.379599 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [req-a52e134a-5c7a-4389-85d0-80f21789b79b req-5553c8e1-b9e4-42b7-b2b9-a8cc157b315f service nova] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:25:56.391304 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.compute.server_external_events [req-a52e134a-5c7a-4389-85d0-80f21789b79b req-5553c8e1-b9e4-42b7-b2b9-a8cc157b315f service nova] Creating event network-vif-plugged:89f4d982-1807-4e80-b9ef-c7bc0025f1b0 for instance 9941c49f-8626-461d-a570-c0ffbfd4b804 on hv-cinder-84630 Aug 27 02:25:56.396445 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [req-a52e134a-5c7a-4389-85d0-80f21789b79b req-5553c8e1-b9e4-42b7-b2b9-a8cc157b315f service nova] 10.222.0.52 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.035046 Aug 27 02:25:56.397224 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 236/472] 10.222.0.52 () {64 vars in 1317 bytes} [Sat Aug 27 02:25:56 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 36 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 27 02:25:56.567269 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-8f45c55d-a0cc-472a-9b13-0fe63efc7cca tempest-VolumesSnapshotTestJSON-1750456804 tempest-VolumesSnapshotTestJSON-1750456804-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:25:56.567894 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-8f45c55d-a0cc-472a-9b13-0fe63efc7cca tempest-VolumesSnapshotTestJSON-1750456804 tempest-VolumesSnapshotTestJSON-1750456804-project] [instance: c31c26a8-fe5a-40fb-91a1-7b5251fd0daa] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:25:56.569973 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG dbcounter [-] [120863] Writing DB stats nova_cell0:SELECT=12 {{(pid=120863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 27 02:25:56.575495 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-8f45c55d-a0cc-472a-9b13-0fe63efc7cca tempest-VolumesSnapshotTestJSON-1750456804 tempest-VolumesSnapshotTestJSON-1750456804-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:25:56.576295 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-8f45c55d-a0cc-472a-9b13-0fe63efc7cca tempest-VolumesSnapshotTestJSON-1750456804 tempest-VolumesSnapshotTestJSON-1750456804-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:25:56.576736 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-8f45c55d-a0cc-472a-9b13-0fe63efc7cca tempest-VolumesSnapshotTestJSON-1750456804 tempest-VolumesSnapshotTestJSON-1750456804-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:25:56.577037 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-26ed77c4-4bc7-43e7-9510-4ba867f9b616 tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Calling method '>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:25:56.577604 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-8f45c55d-a0cc-472a-9b13-0fe63efc7cca tempest-VolumesSnapshotTestJSON-1750456804 tempest-VolumesSnapshotTestJSON-1750456804-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:25:56.577932 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-26ed77c4-4bc7-43e7-9510-4ba867f9b616 tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] [instance: fbea9949-6c55-4ae7-96e1-b09f1a8030f1] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:25:56.578064 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-8f45c55d-a0cc-472a-9b13-0fe63efc7cca tempest-VolumesSnapshotTestJSON-1750456804 tempest-VolumesSnapshotTestJSON-1750456804-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:25:56.578495 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-8f45c55d-a0cc-472a-9b13-0fe63efc7cca tempest-VolumesSnapshotTestJSON-1750456804 tempest-VolumesSnapshotTestJSON-1750456804-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:25:56.584479 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-26ed77c4-4bc7-43e7-9510-4ba867f9b616 tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:25:56.584996 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-26ed77c4-4bc7-43e7-9510-4ba867f9b616 tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:25:56.585474 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-26ed77c4-4bc7-43e7-9510-4ba867f9b616 tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:25:56.586169 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-26ed77c4-4bc7-43e7-9510-4ba867f9b616 tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:25:56.586615 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-26ed77c4-4bc7-43e7-9510-4ba867f9b616 tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:25:56.587034 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-26ed77c4-4bc7-43e7-9510-4ba867f9b616 tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:25:56.799254 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-8f45c55d-a0cc-472a-9b13-0fe63efc7cca tempest-VolumesSnapshotTestJSON-1750456804 tempest-VolumesSnapshotTestJSON-1750456804-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '75744eccc3d24d4c9282cbe3a3314489', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fa43d54225bf44b283f9db5ddbbb13ca', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:25:56.803805 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-26ed77c4-4bc7-43e7-9510-4ba867f9b616 tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2e60cf2931f04f0e8e911adea32979dd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7cb3e3496c1a45c8942c3678c0c299e5', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:25:56.807867 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-8f45c55d-a0cc-472a-9b13-0fe63efc7cca tempest-VolumesSnapshotTestJSON-1750456804 tempest-VolumesSnapshotTestJSON-1750456804-project] 10.222.0.52 "GET /compute/v2.1/servers/c31c26a8-fe5a-40fb-91a1-7b5251fd0daa" status: 200 len: 1319 microversion: 2.1 time: 0.245472 Aug 27 02:25:56.808447 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 237/473] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:25:56 2022] GET /compute/v2.1/servers/c31c26a8-fe5a-40fb-91a1-7b5251fd0daa => generated 1319 bytes in 247 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:25:56.812876 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-26ed77c4-4bc7-43e7-9510-4ba867f9b616 tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] 10.222.0.52 "GET /compute/v2.1/servers/fbea9949-6c55-4ae7-96e1-b09f1a8030f1" status: 200 len: 1314 microversion: 2.1 time: 0.240121 Aug 27 02:25:56.813519 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 237/474] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:25:56 2022] GET /compute/v2.1/servers/fbea9949-6c55-4ae7-96e1-b09f1a8030f1 => generated 1314 bytes in 242 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 27 02:25:56.876903 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-014e7110-7d5d-445e-aedf-cb13ae3f6777 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:25:56.877613 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-014e7110-7d5d-445e-aedf-cb13ae3f6777 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] [instance: 98e4cfeb-d49e-4888-a2c0-40a9e8af90d3] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:25:56.883987 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-014e7110-7d5d-445e-aedf-cb13ae3f6777 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:25:56.884468 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-014e7110-7d5d-445e-aedf-cb13ae3f6777 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:25:56.884899 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-014e7110-7d5d-445e-aedf-cb13ae3f6777 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:25:56.885624 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-014e7110-7d5d-445e-aedf-cb13ae3f6777 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:25:56.886070 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-014e7110-7d5d-445e-aedf-cb13ae3f6777 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:25:56.886480 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-014e7110-7d5d-445e-aedf-cb13ae3f6777 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:25:56.979366 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-0132b121-801c-42bd-ba4b-f1724912a6fc tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] Calling method '>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:25:56.979967 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-0132b121-801c-42bd-ba4b-f1724912a6fc tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] [instance: 9941c49f-8626-461d-a570-c0ffbfd4b804] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:25:56.986332 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-0132b121-801c-42bd-ba4b-f1724912a6fc tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:25:56.987038 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-0132b121-801c-42bd-ba4b-f1724912a6fc tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:25:56.987448 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-0132b121-801c-42bd-ba4b-f1724912a6fc tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:25:56.988631 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-0132b121-801c-42bd-ba4b-f1724912a6fc tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:25:56.989109 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-0132b121-801c-42bd-ba4b-f1724912a6fc tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:25:56.989521 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-0132b121-801c-42bd-ba4b-f1724912a6fc tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:25:57.046632 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-014e7110-7d5d-445e-aedf-cb13ae3f6777 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd945d512ae034ec08838d0a3afde2429', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '880ac4003248441c809a76cfd1b221eb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:25:57.054914 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-014e7110-7d5d-445e-aedf-cb13ae3f6777 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] 10.222.0.52 "GET /compute/v2.1/servers/98e4cfeb-d49e-4888-a2c0-40a9e8af90d3" status: 200 len: 1370 microversion: 2.1 time: 0.182039 Aug 27 02:25:57.055421 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 238/475] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:25:56 2022] GET /compute/v2.1/servers/98e4cfeb-d49e-4888-a2c0-40a9e8af90d3 => generated 1370 bytes in 183 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:25:57.137702 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-0132b121-801c-42bd-ba4b-f1724912a6fc tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '642291c4daa04a6ab3fbce92a8be39c6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b2ddff853b7546788b645a0344c2297b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:25:57.146179 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-0132b121-801c-42bd-ba4b-f1724912a6fc tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] 10.222.0.52 "GET /compute/v2.1/servers/9941c49f-8626-461d-a570-c0ffbfd4b804" status: 200 len: 1260 microversion: 2.1 time: 0.170609 Aug 27 02:25:57.146779 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 238/476] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:25:56 2022] GET /compute/v2.1/servers/9941c49f-8626-461d-a570-c0ffbfd4b804 => generated 1260 bytes in 172 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 27 02:25:57.830076 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-83b9802d-7f85-4940-a427-981538690b0a tempest-VolumesSnapshotTestJSON-1750456804 tempest-VolumesSnapshotTestJSON-1750456804-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:25:57.830674 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-83b9802d-7f85-4940-a427-981538690b0a tempest-VolumesSnapshotTestJSON-1750456804 tempest-VolumesSnapshotTestJSON-1750456804-project] [instance: c31c26a8-fe5a-40fb-91a1-7b5251fd0daa] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:25:57.834653 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-7911961e-3ce7-4134-8fc1-a738a4874995 tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Calling method '>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:25:57.835306 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-7911961e-3ce7-4134-8fc1-a738a4874995 tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] [instance: fbea9949-6c55-4ae7-96e1-b09f1a8030f1] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:25:57.838491 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-83b9802d-7f85-4940-a427-981538690b0a tempest-VolumesSnapshotTestJSON-1750456804 tempest-VolumesSnapshotTestJSON-1750456804-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:25:57.838980 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-83b9802d-7f85-4940-a427-981538690b0a tempest-VolumesSnapshotTestJSON-1750456804 tempest-VolumesSnapshotTestJSON-1750456804-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:25:57.839412 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-83b9802d-7f85-4940-a427-981538690b0a tempest-VolumesSnapshotTestJSON-1750456804 tempest-VolumesSnapshotTestJSON-1750456804-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:25:57.840141 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-83b9802d-7f85-4940-a427-981538690b0a tempest-VolumesSnapshotTestJSON-1750456804 tempest-VolumesSnapshotTestJSON-1750456804-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:25:57.840593 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-83b9802d-7f85-4940-a427-981538690b0a tempest-VolumesSnapshotTestJSON-1750456804 tempest-VolumesSnapshotTestJSON-1750456804-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:25:57.841016 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-83b9802d-7f85-4940-a427-981538690b0a tempest-VolumesSnapshotTestJSON-1750456804 tempest-VolumesSnapshotTestJSON-1750456804-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:25:57.841208 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-7911961e-3ce7-4134-8fc1-a738a4874995 tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:25:57.841690 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-7911961e-3ce7-4134-8fc1-a738a4874995 tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:25:57.842122 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-7911961e-3ce7-4134-8fc1-a738a4874995 tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:25:57.842809 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-7911961e-3ce7-4134-8fc1-a738a4874995 tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:25:57.843258 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-7911961e-3ce7-4134-8fc1-a738a4874995 tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:25:57.843673 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-7911961e-3ce7-4134-8fc1-a738a4874995 tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:25:57.845221 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG dbcounter [-] [120863] Writing DB stats nova_cell1:SELECT=100 {{(pid=120863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 27 02:25:58.037035 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-7911961e-3ce7-4134-8fc1-a738a4874995 tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2e60cf2931f04f0e8e911adea32979dd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7cb3e3496c1a45c8942c3678c0c299e5', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:25:58.040720 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG dbcounter [-] [120864] Writing DB stats nova_cell1:SELECT=100 {{(pid=120864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 27 02:25:58.045633 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-7911961e-3ce7-4134-8fc1-a738a4874995 tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] 10.222.0.52 "GET /compute/v2.1/servers/fbea9949-6c55-4ae7-96e1-b09f1a8030f1" status: 200 len: 1314 microversion: 2.1 time: 0.214790 Aug 27 02:25:58.046196 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 239/477] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:25:57 2022] GET /compute/v2.1/servers/fbea9949-6c55-4ae7-96e1-b09f1a8030f1 => generated 1314 bytes in 216 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:25:58.049775 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-83b9802d-7f85-4940-a427-981538690b0a tempest-VolumesSnapshotTestJSON-1750456804 tempest-VolumesSnapshotTestJSON-1750456804-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '75744eccc3d24d4c9282cbe3a3314489', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fa43d54225bf44b283f9db5ddbbb13ca', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:25:58.058160 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-83b9802d-7f85-4940-a427-981538690b0a tempest-VolumesSnapshotTestJSON-1750456804 tempest-VolumesSnapshotTestJSON-1750456804-project] 10.222.0.52 "GET /compute/v2.1/servers/c31c26a8-fe5a-40fb-91a1-7b5251fd0daa" status: 200 len: 1319 microversion: 2.1 time: 0.232362 Aug 27 02:25:58.058913 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 239/478] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:25:57 2022] GET /compute/v2.1/servers/c31c26a8-fe5a-40fb-91a1-7b5251fd0daa => generated 1319 bytes in 234 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:25:58.078406 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-16c41bb1-1998-4725-a370-17d5cffe4ac4 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Calling method '>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:25:58.079260 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-16c41bb1-1998-4725-a370-17d5cffe4ac4 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] [instance: 98e4cfeb-d49e-4888-a2c0-40a9e8af90d3] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:25:58.086463 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-16c41bb1-1998-4725-a370-17d5cffe4ac4 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:25:58.086902 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-16c41bb1-1998-4725-a370-17d5cffe4ac4 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:25:58.087336 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-16c41bb1-1998-4725-a370-17d5cffe4ac4 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:25:58.088210 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-16c41bb1-1998-4725-a370-17d5cffe4ac4 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:25:58.088478 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-16c41bb1-1998-4725-a370-17d5cffe4ac4 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:25:58.089064 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-16c41bb1-1998-4725-a370-17d5cffe4ac4 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:25:58.168138 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-cfdb8f38-8d09-478b-b7be-0ebe07113391 tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:25:58.168913 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-cfdb8f38-8d09-478b-b7be-0ebe07113391 tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] [instance: 9941c49f-8626-461d-a570-c0ffbfd4b804] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:25:58.175315 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-cfdb8f38-8d09-478b-b7be-0ebe07113391 tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:25:58.176224 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-cfdb8f38-8d09-478b-b7be-0ebe07113391 tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:25:58.176675 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-cfdb8f38-8d09-478b-b7be-0ebe07113391 tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:25:58.177625 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-cfdb8f38-8d09-478b-b7be-0ebe07113391 tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:25:58.178237 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-cfdb8f38-8d09-478b-b7be-0ebe07113391 tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:25:58.178656 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-cfdb8f38-8d09-478b-b7be-0ebe07113391 tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:25:58.241325 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-16c41bb1-1998-4725-a370-17d5cffe4ac4 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd945d512ae034ec08838d0a3afde2429', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '880ac4003248441c809a76cfd1b221eb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:25:58.249633 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-16c41bb1-1998-4725-a370-17d5cffe4ac4 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] 10.222.0.52 "GET /compute/v2.1/servers/98e4cfeb-d49e-4888-a2c0-40a9e8af90d3" status: 200 len: 1370 microversion: 2.1 time: 0.175935 Aug 27 02:25:58.250220 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 240/479] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:25:58 2022] GET /compute/v2.1/servers/98e4cfeb-d49e-4888-a2c0-40a9e8af90d3 => generated 1370 bytes in 177 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:25:58.339428 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-cfdb8f38-8d09-478b-b7be-0ebe07113391 tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '642291c4daa04a6ab3fbce92a8be39c6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b2ddff853b7546788b645a0344c2297b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:25:58.347074 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-cfdb8f38-8d09-478b-b7be-0ebe07113391 tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] 10.222.0.52 "GET /compute/v2.1/servers/9941c49f-8626-461d-a570-c0ffbfd4b804" status: 200 len: 1260 microversion: 2.1 time: 0.183707 Aug 27 02:25:58.347736 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 240/480] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:25:58 2022] GET /compute/v2.1/servers/9941c49f-8626-461d-a570-c0ffbfd4b804 => generated 1260 bytes in 185 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 27 02:25:59.067162 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-f3f6c777-d63a-4189-854d-32206bc8bbfb tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Calling method '>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:25:59.067804 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-f3f6c777-d63a-4189-854d-32206bc8bbfb tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] [instance: fbea9949-6c55-4ae7-96e1-b09f1a8030f1] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:25:59.074112 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-f3f6c777-d63a-4189-854d-32206bc8bbfb tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:25:59.074690 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-f3f6c777-d63a-4189-854d-32206bc8bbfb tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:25:59.075200 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-f3f6c777-d63a-4189-854d-32206bc8bbfb tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:25:59.076237 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-f3f6c777-d63a-4189-854d-32206bc8bbfb tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:25:59.076759 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-f3f6c777-d63a-4189-854d-32206bc8bbfb tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:25:59.077318 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-f3f6c777-d63a-4189-854d-32206bc8bbfb tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:25:59.083007 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-36fb398e-f803-497c-ac76-b434f23f4c6d tempest-VolumesSnapshotTestJSON-1750456804 tempest-VolumesSnapshotTestJSON-1750456804-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:25:59.083718 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-36fb398e-f803-497c-ac76-b434f23f4c6d tempest-VolumesSnapshotTestJSON-1750456804 tempest-VolumesSnapshotTestJSON-1750456804-project] [instance: c31c26a8-fe5a-40fb-91a1-7b5251fd0daa] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:25:59.090675 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-36fb398e-f803-497c-ac76-b434f23f4c6d tempest-VolumesSnapshotTestJSON-1750456804 tempest-VolumesSnapshotTestJSON-1750456804-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:25:59.091303 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-36fb398e-f803-497c-ac76-b434f23f4c6d tempest-VolumesSnapshotTestJSON-1750456804 tempest-VolumesSnapshotTestJSON-1750456804-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:25:59.091744 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-36fb398e-f803-497c-ac76-b434f23f4c6d tempest-VolumesSnapshotTestJSON-1750456804 tempest-VolumesSnapshotTestJSON-1750456804-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:25:59.092531 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-36fb398e-f803-497c-ac76-b434f23f4c6d tempest-VolumesSnapshotTestJSON-1750456804 tempest-VolumesSnapshotTestJSON-1750456804-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:25:59.093077 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-36fb398e-f803-497c-ac76-b434f23f4c6d tempest-VolumesSnapshotTestJSON-1750456804 tempest-VolumesSnapshotTestJSON-1750456804-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:25:59.093737 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-36fb398e-f803-497c-ac76-b434f23f4c6d tempest-VolumesSnapshotTestJSON-1750456804 tempest-VolumesSnapshotTestJSON-1750456804-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:25:59.275153 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-f3f6c777-d63a-4189-854d-32206bc8bbfb tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2e60cf2931f04f0e8e911adea32979dd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7cb3e3496c1a45c8942c3678c0c299e5', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:25:59.283166 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-f3f6c777-d63a-4189-854d-32206bc8bbfb tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] 10.222.0.52 "GET /compute/v2.1/servers/fbea9949-6c55-4ae7-96e1-b09f1a8030f1" status: 200 len: 1314 microversion: 2.1 time: 0.220402 Aug 27 02:25:59.283710 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 241/481] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:25:59 2022] GET /compute/v2.1/servers/fbea9949-6c55-4ae7-96e1-b09f1a8030f1 => generated 1314 bytes in 222 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:25:59.287526 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-36fb398e-f803-497c-ac76-b434f23f4c6d tempest-VolumesSnapshotTestJSON-1750456804 tempest-VolumesSnapshotTestJSON-1750456804-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '75744eccc3d24d4c9282cbe3a3314489', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fa43d54225bf44b283f9db5ddbbb13ca', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:25:59.288240 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-e6ea9607-e89e-4dd5-a5f6-e569c6011131 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Calling method '>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:25:59.288830 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-e6ea9607-e89e-4dd5-a5f6-e569c6011131 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] [instance: 98e4cfeb-d49e-4888-a2c0-40a9e8af90d3] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:25:59.294916 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-e6ea9607-e89e-4dd5-a5f6-e569c6011131 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:25:59.295450 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-e6ea9607-e89e-4dd5-a5f6-e569c6011131 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:25:59.295879 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-e6ea9607-e89e-4dd5-a5f6-e569c6011131 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:25:59.296516 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-36fb398e-f803-497c-ac76-b434f23f4c6d tempest-VolumesSnapshotTestJSON-1750456804 tempest-VolumesSnapshotTestJSON-1750456804-project] 10.222.0.52 "GET /compute/v2.1/servers/c31c26a8-fe5a-40fb-91a1-7b5251fd0daa" status: 200 len: 1319 microversion: 2.1 time: 0.217381 Aug 27 02:25:59.296621 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-e6ea9607-e89e-4dd5-a5f6-e569c6011131 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:25:59.296988 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-e6ea9607-e89e-4dd5-a5f6-e569c6011131 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:25:59.297137 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 241/482] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:25:59 2022] GET /compute/v2.1/servers/c31c26a8-fe5a-40fb-91a1-7b5251fd0daa => generated 1319 bytes in 219 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 27 02:25:59.297496 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-e6ea9607-e89e-4dd5-a5f6-e569c6011131 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:25:59.368286 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-8100531e-d0b8-41b5-9f3e-80c8acbf40d9 tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:25:59.368998 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-8100531e-d0b8-41b5-9f3e-80c8acbf40d9 tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] [instance: 9941c49f-8626-461d-a570-c0ffbfd4b804] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:25:59.375286 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-8100531e-d0b8-41b5-9f3e-80c8acbf40d9 tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:25:59.375769 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-8100531e-d0b8-41b5-9f3e-80c8acbf40d9 tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:25:59.376208 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-8100531e-d0b8-41b5-9f3e-80c8acbf40d9 tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:25:59.376879 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-8100531e-d0b8-41b5-9f3e-80c8acbf40d9 tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:25:59.377363 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-8100531e-d0b8-41b5-9f3e-80c8acbf40d9 tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:25:59.377779 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-8100531e-d0b8-41b5-9f3e-80c8acbf40d9 tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:25:59.460388 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-e6ea9607-e89e-4dd5-a5f6-e569c6011131 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd945d512ae034ec08838d0a3afde2429', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '880ac4003248441c809a76cfd1b221eb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:25:59.469182 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-e6ea9607-e89e-4dd5-a5f6-e569c6011131 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] 10.222.0.52 "GET /compute/v2.1/servers/98e4cfeb-d49e-4888-a2c0-40a9e8af90d3" status: 200 len: 1370 microversion: 2.1 time: 0.184437 Aug 27 02:25:59.469782 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 242/483] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:25:59 2022] GET /compute/v2.1/servers/98e4cfeb-d49e-4888-a2c0-40a9e8af90d3 => generated 1370 bytes in 185 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:25:59.532182 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-8100531e-d0b8-41b5-9f3e-80c8acbf40d9 tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '642291c4daa04a6ab3fbce92a8be39c6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b2ddff853b7546788b645a0344c2297b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:25:59.540556 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-8100531e-d0b8-41b5-9f3e-80c8acbf40d9 tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] 10.222.0.52 "GET /compute/v2.1/servers/9941c49f-8626-461d-a570-c0ffbfd4b804" status: 200 len: 1260 microversion: 2.1 time: 0.176174 Aug 27 02:25:59.541149 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 242/484] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:25:59 2022] GET /compute/v2.1/servers/9941c49f-8626-461d-a570-c0ffbfd4b804 => generated 1260 bytes in 178 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:26:00.304709 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-79a96cc4-33b3-4ae9-9465-07fae64a3a73 tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Calling method '>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:26:00.305512 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-79a96cc4-33b3-4ae9-9465-07fae64a3a73 tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] [instance: fbea9949-6c55-4ae7-96e1-b09f1a8030f1] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:26:00.311913 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-79a96cc4-33b3-4ae9-9465-07fae64a3a73 tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:26:00.312402 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-79a96cc4-33b3-4ae9-9465-07fae64a3a73 tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:26:00.312810 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-79a96cc4-33b3-4ae9-9465-07fae64a3a73 tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:26:00.313597 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-79a96cc4-33b3-4ae9-9465-07fae64a3a73 tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:26:00.314032 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-79a96cc4-33b3-4ae9-9465-07fae64a3a73 tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:26:00.314468 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-79a96cc4-33b3-4ae9-9465-07fae64a3a73 tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:26:00.319669 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-b65b9ede-e017-47ce-af7f-7b1d8a9750e6 tempest-VolumesSnapshotTestJSON-1750456804 tempest-VolumesSnapshotTestJSON-1750456804-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:26:00.320369 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-b65b9ede-e017-47ce-af7f-7b1d8a9750e6 tempest-VolumesSnapshotTestJSON-1750456804 tempest-VolumesSnapshotTestJSON-1750456804-project] [instance: c31c26a8-fe5a-40fb-91a1-7b5251fd0daa] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:26:00.327628 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-b65b9ede-e017-47ce-af7f-7b1d8a9750e6 tempest-VolumesSnapshotTestJSON-1750456804 tempest-VolumesSnapshotTestJSON-1750456804-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:26:00.329093 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-b65b9ede-e017-47ce-af7f-7b1d8a9750e6 tempest-VolumesSnapshotTestJSON-1750456804 tempest-VolumesSnapshotTestJSON-1750456804-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:26:00.329787 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-b65b9ede-e017-47ce-af7f-7b1d8a9750e6 tempest-VolumesSnapshotTestJSON-1750456804 tempest-VolumesSnapshotTestJSON-1750456804-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:26:00.330857 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-b65b9ede-e017-47ce-af7f-7b1d8a9750e6 tempest-VolumesSnapshotTestJSON-1750456804 tempest-VolumesSnapshotTestJSON-1750456804-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:26:00.331523 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-b65b9ede-e017-47ce-af7f-7b1d8a9750e6 tempest-VolumesSnapshotTestJSON-1750456804 tempest-VolumesSnapshotTestJSON-1750456804-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:26:00.332571 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-b65b9ede-e017-47ce-af7f-7b1d8a9750e6 tempest-VolumesSnapshotTestJSON-1750456804 tempest-VolumesSnapshotTestJSON-1750456804-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:26:00.532781 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-79a96cc4-33b3-4ae9-9465-07fae64a3a73 tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2e60cf2931f04f0e8e911adea32979dd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7cb3e3496c1a45c8942c3678c0c299e5', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:26:00.540234 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-b65b9ede-e017-47ce-af7f-7b1d8a9750e6 tempest-VolumesSnapshotTestJSON-1750456804 tempest-VolumesSnapshotTestJSON-1750456804-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '75744eccc3d24d4c9282cbe3a3314489', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fa43d54225bf44b283f9db5ddbbb13ca', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:26:00.541759 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-79a96cc4-33b3-4ae9-9465-07fae64a3a73 tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] 10.222.0.52 "GET /compute/v2.1/servers/fbea9949-6c55-4ae7-96e1-b09f1a8030f1" status: 200 len: 1314 microversion: 2.1 time: 0.241078 Aug 27 02:26:00.542534 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 243/485] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:26:00 2022] GET /compute/v2.1/servers/fbea9949-6c55-4ae7-96e1-b09f1a8030f1 => generated 1314 bytes in 243 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:26:00.547125 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [req-63b4a8c9-568c-4aab-a5bd-0644233e7ce9 req-d9b1246b-2f19-4290-81b0-d48de8a5a912 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "fbea9949-6c55-4ae7-96e1-b09f1a8030f1", "name": "network-vif-plugged", "status": "completed", "tag": "0d4a34a6-1648-44be-9eb9-79840124e8e3"}]} {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 27 02:26:00.548625 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-b65b9ede-e017-47ce-af7f-7b1d8a9750e6 tempest-VolumesSnapshotTestJSON-1750456804 tempest-VolumesSnapshotTestJSON-1750456804-project] 10.222.0.52 "GET /compute/v2.1/servers/c31c26a8-fe5a-40fb-91a1-7b5251fd0daa" status: 200 len: 1508 microversion: 2.1 time: 0.233741 Aug 27 02:26:00.557348 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [req-63b4a8c9-568c-4aab-a5bd-0644233e7ce9 req-d9b1246b-2f19-4290-81b0-d48de8a5a912 service nova] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:26:00.557830 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [req-63b4a8c9-568c-4aab-a5bd-0644233e7ce9 req-d9b1246b-2f19-4290-81b0-d48de8a5a912 service nova] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:26:00.558259 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [req-63b4a8c9-568c-4aab-a5bd-0644233e7ce9 req-d9b1246b-2f19-4290-81b0-d48de8a5a912 service nova] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:26:00.565552 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 243/486] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:26:00 2022] GET /compute/v2.1/servers/c31c26a8-fe5a-40fb-91a1-7b5251fd0daa => generated 1508 bytes in 251 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:26:00.570535 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-9caafe91-ce10-48a7-b3ba-547933d392fc tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:26:00.571148 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-9caafe91-ce10-48a7-b3ba-547933d392fc tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] [instance: 98e4cfeb-d49e-4888-a2c0-40a9e8af90d3] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:26:00.571255 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.compute.server_external_events [req-63b4a8c9-568c-4aab-a5bd-0644233e7ce9 req-d9b1246b-2f19-4290-81b0-d48de8a5a912 service nova] Creating event network-vif-plugged:0d4a34a6-1648-44be-9eb9-79840124e8e3 for instance fbea9949-6c55-4ae7-96e1-b09f1a8030f1 on hv-cinder-84630 Aug 27 02:26:00.576782 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [req-63b4a8c9-568c-4aab-a5bd-0644233e7ce9 req-d9b1246b-2f19-4290-81b0-d48de8a5a912 service nova] 10.222.0.52 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.032564 Aug 27 02:26:00.577704 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 244/487] 10.222.0.52 () {64 vars in 1317 bytes} [Sat Aug 27 02:26:00 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 34 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 27 02:26:00.577837 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-9caafe91-ce10-48a7-b3ba-547933d392fc tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:26:00.578323 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-9caafe91-ce10-48a7-b3ba-547933d392fc tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:26:00.578760 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-9caafe91-ce10-48a7-b3ba-547933d392fc tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:26:00.579670 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-9caafe91-ce10-48a7-b3ba-547933d392fc tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:26:00.580122 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-9caafe91-ce10-48a7-b3ba-547933d392fc tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:26:00.580547 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-9caafe91-ce10-48a7-b3ba-547933d392fc tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:26:00.586410 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-1b7e6915-fe7f-40a0-af95-88acb9e7a3ba tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] Calling method '>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:26:00.587113 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-1b7e6915-fe7f-40a0-af95-88acb9e7a3ba tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] [instance: 9941c49f-8626-461d-a570-c0ffbfd4b804] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:26:00.593855 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-1b7e6915-fe7f-40a0-af95-88acb9e7a3ba tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:26:00.594340 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-1b7e6915-fe7f-40a0-af95-88acb9e7a3ba tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:26:00.594979 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-1b7e6915-fe7f-40a0-af95-88acb9e7a3ba tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:26:00.595683 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-1b7e6915-fe7f-40a0-af95-88acb9e7a3ba tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:26:00.596131 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-1b7e6915-fe7f-40a0-af95-88acb9e7a3ba tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:26:00.596548 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-1b7e6915-fe7f-40a0-af95-88acb9e7a3ba tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:26:00.813980 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-1b7e6915-fe7f-40a0-af95-88acb9e7a3ba tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '642291c4daa04a6ab3fbce92a8be39c6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b2ddff853b7546788b645a0344c2297b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:26:00.819299 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-9caafe91-ce10-48a7-b3ba-547933d392fc tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd945d512ae034ec08838d0a3afde2429', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '880ac4003248441c809a76cfd1b221eb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:26:00.822488 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-1b7e6915-fe7f-40a0-af95-88acb9e7a3ba tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] 10.222.0.52 "GET /compute/v2.1/servers/9941c49f-8626-461d-a570-c0ffbfd4b804" status: 200 len: 1260 microversion: 2.1 time: 0.240977 Aug 27 02:26:00.823144 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 245/488] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:26:00 2022] GET /compute/v2.1/servers/9941c49f-8626-461d-a570-c0ffbfd4b804 => generated 1260 bytes in 245 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:26:00.834298 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-9caafe91-ce10-48a7-b3ba-547933d392fc tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] 10.222.0.52 "GET /compute/v2.1/servers/98e4cfeb-d49e-4888-a2c0-40a9e8af90d3" status: 200 len: 1370 microversion: 2.1 time: 0.267424 Aug 27 02:26:00.835192 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 244/489] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:26:00 2022] GET /compute/v2.1/servers/98e4cfeb-d49e-4888-a2c0-40a9e8af90d3 => generated 1370 bytes in 269 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:26:01.564334 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-4b24c761-3931-4830-892a-eaf4dc01e1a3 tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Calling method '>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:26:01.564913 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-4b24c761-3931-4830-892a-eaf4dc01e1a3 tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] [instance: fbea9949-6c55-4ae7-96e1-b09f1a8030f1] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:26:01.572340 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-4b24c761-3931-4830-892a-eaf4dc01e1a3 tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:26:01.572822 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-4b24c761-3931-4830-892a-eaf4dc01e1a3 tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:26:01.573282 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-4b24c761-3931-4830-892a-eaf4dc01e1a3 tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:26:01.573963 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-4b24c761-3931-4830-892a-eaf4dc01e1a3 tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:26:01.574406 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-4b24c761-3931-4830-892a-eaf4dc01e1a3 tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:26:01.574817 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-4b24c761-3931-4830-892a-eaf4dc01e1a3 tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:26:01.737801 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-4b24c761-3931-4830-892a-eaf4dc01e1a3 tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2e60cf2931f04f0e8e911adea32979dd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7cb3e3496c1a45c8942c3678c0c299e5', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:26:01.746438 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-4b24c761-3931-4830-892a-eaf4dc01e1a3 tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] 10.222.0.52 "GET /compute/v2.1/servers/fbea9949-6c55-4ae7-96e1-b09f1a8030f1" status: 200 len: 1314 microversion: 2.1 time: 0.186089 Aug 27 02:26:01.746983 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 246/490] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:26:01 2022] GET /compute/v2.1/servers/fbea9949-6c55-4ae7-96e1-b09f1a8030f1 => generated 1314 bytes in 187 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:26:01.844411 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-ab728eb3-d478-4019-86c4-eb511276c2d5 tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:26:01.845003 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-ab728eb3-d478-4019-86c4-eb511276c2d5 tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] [instance: 9941c49f-8626-461d-a570-c0ffbfd4b804] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:26:01.853074 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-ab728eb3-d478-4019-86c4-eb511276c2d5 tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:26:01.853676 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-ab728eb3-d478-4019-86c4-eb511276c2d5 tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:26:01.854176 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-ab728eb3-d478-4019-86c4-eb511276c2d5 tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:26:01.854937 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-ab728eb3-d478-4019-86c4-eb511276c2d5 tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:26:01.855071 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-e8e18b24-bf31-4fee-869c-2ecdb8afc205 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Calling method '>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:26:01.855655 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-e8e18b24-bf31-4fee-869c-2ecdb8afc205 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] [instance: 98e4cfeb-d49e-4888-a2c0-40a9e8af90d3] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:26:01.855810 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-ab728eb3-d478-4019-86c4-eb511276c2d5 tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:26:01.856398 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-ab728eb3-d478-4019-86c4-eb511276c2d5 tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:26:01.863012 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-e8e18b24-bf31-4fee-869c-2ecdb8afc205 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:26:01.864022 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-e8e18b24-bf31-4fee-869c-2ecdb8afc205 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:26:01.865075 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-e8e18b24-bf31-4fee-869c-2ecdb8afc205 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:26:01.866016 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-e8e18b24-bf31-4fee-869c-2ecdb8afc205 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:26:01.867179 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-e8e18b24-bf31-4fee-869c-2ecdb8afc205 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:26:01.867949 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-e8e18b24-bf31-4fee-869c-2ecdb8afc205 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:26:02.124719 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-ab728eb3-d478-4019-86c4-eb511276c2d5 tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '642291c4daa04a6ab3fbce92a8be39c6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b2ddff853b7546788b645a0344c2297b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:26:02.131455 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-e8e18b24-bf31-4fee-869c-2ecdb8afc205 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd945d512ae034ec08838d0a3afde2429', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '880ac4003248441c809a76cfd1b221eb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:26:02.136618 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-ab728eb3-d478-4019-86c4-eb511276c2d5 tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] 10.222.0.52 "GET /compute/v2.1/servers/9941c49f-8626-461d-a570-c0ffbfd4b804" status: 200 len: 1444 microversion: 2.1 time: 0.296404 Aug 27 02:26:02.137318 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 245/491] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:26:01 2022] GET /compute/v2.1/servers/9941c49f-8626-461d-a570-c0ffbfd4b804 => generated 1444 bytes in 298 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:26:02.139997 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-e8e18b24-bf31-4fee-869c-2ecdb8afc205 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] 10.222.0.52 "GET /compute/v2.1/servers/98e4cfeb-d49e-4888-a2c0-40a9e8af90d3" status: 200 len: 1370 microversion: 2.1 time: 0.288399 Aug 27 02:26:02.140660 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 247/492] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:26:01 2022] GET /compute/v2.1/servers/98e4cfeb-d49e-4888-a2c0-40a9e8af90d3 => generated 1370 bytes in 290 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:26:02.164490 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-147f1965-e02f-4846-bf4e-97446eaf9fa1 tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:26:02.165478 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-147f1965-e02f-4846-bf4e-97446eaf9fa1 tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] [instance: 9941c49f-8626-461d-a570-c0ffbfd4b804] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:26:02.173078 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-147f1965-e02f-4846-bf4e-97446eaf9fa1 tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:26:02.173617 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-147f1965-e02f-4846-bf4e-97446eaf9fa1 tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:26:02.174160 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-147f1965-e02f-4846-bf4e-97446eaf9fa1 tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:26:02.174891 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-147f1965-e02f-4846-bf4e-97446eaf9fa1 tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:26:02.178944 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-147f1965-e02f-4846-bf4e-97446eaf9fa1 tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:26:02.178944 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-147f1965-e02f-4846-bf4e-97446eaf9fa1 tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:26:02.327999 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-147f1965-e02f-4846-bf4e-97446eaf9fa1 tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '642291c4daa04a6ab3fbce92a8be39c6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b2ddff853b7546788b645a0344c2297b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:26:02.337496 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-147f1965-e02f-4846-bf4e-97446eaf9fa1 tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] 10.222.0.52 "GET /compute/v2.1/servers/9941c49f-8626-461d-a570-c0ffbfd4b804" status: 200 len: 1444 microversion: 2.1 time: 0.181563 Aug 27 02:26:02.338052 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 246/493] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:26:02 2022] GET /compute/v2.1/servers/9941c49f-8626-461d-a570-c0ffbfd4b804 => generated 1444 bytes in 183 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:26:02.767942 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-e2399295-7386-4ea6-9113-a10d7f00fed1 tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Calling method '>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:26:02.769237 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-e2399295-7386-4ea6-9113-a10d7f00fed1 tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] [instance: fbea9949-6c55-4ae7-96e1-b09f1a8030f1] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:26:02.776221 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-e2399295-7386-4ea6-9113-a10d7f00fed1 tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:26:02.776644 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-e2399295-7386-4ea6-9113-a10d7f00fed1 tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:26:02.777295 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-e2399295-7386-4ea6-9113-a10d7f00fed1 tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:26:02.778106 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-e2399295-7386-4ea6-9113-a10d7f00fed1 tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:26:02.778455 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-e2399295-7386-4ea6-9113-a10d7f00fed1 tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:26:02.779314 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-e2399295-7386-4ea6-9113-a10d7f00fed1 tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:26:02.934047 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-e2399295-7386-4ea6-9113-a10d7f00fed1 tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2e60cf2931f04f0e8e911adea32979dd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7cb3e3496c1a45c8942c3678c0c299e5', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:26:02.942529 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-e2399295-7386-4ea6-9113-a10d7f00fed1 tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] 10.222.0.52 "GET /compute/v2.1/servers/fbea9949-6c55-4ae7-96e1-b09f1a8030f1" status: 200 len: 1314 microversion: 2.1 time: 0.179145 Aug 27 02:26:02.943104 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 248/494] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:26:02 2022] GET /compute/v2.1/servers/fbea9949-6c55-4ae7-96e1-b09f1a8030f1 => generated 1314 bytes in 180 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:26:03.163414 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-d26cf8ba-3fdd-4d1d-8996-ffda5fefcf24 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:26:03.164300 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-d26cf8ba-3fdd-4d1d-8996-ffda5fefcf24 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] [instance: 98e4cfeb-d49e-4888-a2c0-40a9e8af90d3] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:26:03.171780 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-d26cf8ba-3fdd-4d1d-8996-ffda5fefcf24 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:26:03.172256 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-d26cf8ba-3fdd-4d1d-8996-ffda5fefcf24 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:26:03.172708 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-d26cf8ba-3fdd-4d1d-8996-ffda5fefcf24 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:26:03.173469 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-d26cf8ba-3fdd-4d1d-8996-ffda5fefcf24 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:26:03.173988 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-d26cf8ba-3fdd-4d1d-8996-ffda5fefcf24 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:26:03.174415 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-d26cf8ba-3fdd-4d1d-8996-ffda5fefcf24 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:26:03.326330 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-d26cf8ba-3fdd-4d1d-8996-ffda5fefcf24 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd945d512ae034ec08838d0a3afde2429', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '880ac4003248441c809a76cfd1b221eb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:26:03.334984 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-d26cf8ba-3fdd-4d1d-8996-ffda5fefcf24 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] 10.222.0.52 "GET /compute/v2.1/servers/98e4cfeb-d49e-4888-a2c0-40a9e8af90d3" status: 200 len: 1370 microversion: 2.1 time: 0.176906 Aug 27 02:26:03.335766 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 247/495] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:26:03 2022] GET /compute/v2.1/servers/98e4cfeb-d49e-4888-a2c0-40a9e8af90d3 => generated 1370 bytes in 178 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:26:03.963986 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-08d82764-5080-40ed-90fa-0644eac8522a tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Calling method '>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:26:03.964578 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-08d82764-5080-40ed-90fa-0644eac8522a tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] [instance: fbea9949-6c55-4ae7-96e1-b09f1a8030f1] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:26:03.970822 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-08d82764-5080-40ed-90fa-0644eac8522a tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:26:03.971227 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-08d82764-5080-40ed-90fa-0644eac8522a tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:26:03.971656 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-08d82764-5080-40ed-90fa-0644eac8522a tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:26:03.972391 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-08d82764-5080-40ed-90fa-0644eac8522a tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:26:03.972830 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-08d82764-5080-40ed-90fa-0644eac8522a tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:26:03.973296 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-08d82764-5080-40ed-90fa-0644eac8522a tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:26:04.261295 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-08d82764-5080-40ed-90fa-0644eac8522a tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2e60cf2931f04f0e8e911adea32979dd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7cb3e3496c1a45c8942c3678c0c299e5', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:26:04.269060 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-08d82764-5080-40ed-90fa-0644eac8522a tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] 10.222.0.52 "GET /compute/v2.1/servers/fbea9949-6c55-4ae7-96e1-b09f1a8030f1" status: 200 len: 1314 microversion: 2.1 time: 0.308980 Aug 27 02:26:04.270810 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 249/496] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:26:03 2022] GET /compute/v2.1/servers/fbea9949-6c55-4ae7-96e1-b09f1a8030f1 => generated 1314 bytes in 311 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:26:04.355947 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-533eda13-5bfb-4c98-8edb-d49bb12b396e tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:26:04.356747 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-533eda13-5bfb-4c98-8edb-d49bb12b396e tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] [instance: 98e4cfeb-d49e-4888-a2c0-40a9e8af90d3] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:26:04.363856 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-533eda13-5bfb-4c98-8edb-d49bb12b396e tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:26:04.363856 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-533eda13-5bfb-4c98-8edb-d49bb12b396e tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:26:04.364377 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-533eda13-5bfb-4c98-8edb-d49bb12b396e tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:26:04.365039 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-533eda13-5bfb-4c98-8edb-d49bb12b396e tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:26:04.365618 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-533eda13-5bfb-4c98-8edb-d49bb12b396e tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:26:04.366043 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-533eda13-5bfb-4c98-8edb-d49bb12b396e tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:26:04.513794 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-533eda13-5bfb-4c98-8edb-d49bb12b396e tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd945d512ae034ec08838d0a3afde2429', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '880ac4003248441c809a76cfd1b221eb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:26:04.521853 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-533eda13-5bfb-4c98-8edb-d49bb12b396e tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] 10.222.0.52 "GET /compute/v2.1/servers/98e4cfeb-d49e-4888-a2c0-40a9e8af90d3" status: 200 len: 1370 microversion: 2.1 time: 0.169709 Aug 27 02:26:04.531680 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 248/497] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:26:04 2022] GET /compute/v2.1/servers/98e4cfeb-d49e-4888-a2c0-40a9e8af90d3 => generated 1370 bytes in 174 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:26:04.617353 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [req-e492cf27-8f1e-4027-b829-016f7e17fbc4 req-fe5ad5d6-3b7b-44db-8ca6-5bc1a63f0a86 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "9941c49f-8626-461d-a570-c0ffbfd4b804", "tag": "89f4d982-1807-4e80-b9ef-c7bc0025f1b0"}]} {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 27 02:26:04.620067 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [req-e492cf27-8f1e-4027-b829-016f7e17fbc4 req-fe5ad5d6-3b7b-44db-8ca6-5bc1a63f0a86 service nova] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:26:04.620536 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [req-e492cf27-8f1e-4027-b829-016f7e17fbc4 req-fe5ad5d6-3b7b-44db-8ca6-5bc1a63f0a86 service nova] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:26:04.621003 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [req-e492cf27-8f1e-4027-b829-016f7e17fbc4 req-fe5ad5d6-3b7b-44db-8ca6-5bc1a63f0a86 service nova] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:26:04.633446 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.compute.server_external_events [req-e492cf27-8f1e-4027-b829-016f7e17fbc4 req-fe5ad5d6-3b7b-44db-8ca6-5bc1a63f0a86 service nova] Creating event network-changed:89f4d982-1807-4e80-b9ef-c7bc0025f1b0 for instance 9941c49f-8626-461d-a570-c0ffbfd4b804 on hv-cinder-84630 Aug 27 02:26:04.638804 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [req-e492cf27-8f1e-4027-b829-016f7e17fbc4 req-fe5ad5d6-3b7b-44db-8ca6-5bc1a63f0a86 service nova] 10.222.0.52 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.032048 Aug 27 02:26:04.639579 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 250/498] 10.222.0.52 () {64 vars in 1317 bytes} [Sat Aug 27 02:26:04 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 33 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 27 02:26:05.289990 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-afe56292-ff11-4a51-b5cb-5f78cca6d6b3 tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:26:05.290736 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-afe56292-ff11-4a51-b5cb-5f78cca6d6b3 tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] [instance: fbea9949-6c55-4ae7-96e1-b09f1a8030f1] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:26:05.297339 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-afe56292-ff11-4a51-b5cb-5f78cca6d6b3 tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:26:05.297933 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-afe56292-ff11-4a51-b5cb-5f78cca6d6b3 tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:26:05.298465 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-afe56292-ff11-4a51-b5cb-5f78cca6d6b3 tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:26:05.299266 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-afe56292-ff11-4a51-b5cb-5f78cca6d6b3 tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:26:05.299951 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-afe56292-ff11-4a51-b5cb-5f78cca6d6b3 tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:26:05.300520 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-afe56292-ff11-4a51-b5cb-5f78cca6d6b3 tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:26:05.441497 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-afe56292-ff11-4a51-b5cb-5f78cca6d6b3 tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2e60cf2931f04f0e8e911adea32979dd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7cb3e3496c1a45c8942c3678c0c299e5', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:26:05.449381 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-afe56292-ff11-4a51-b5cb-5f78cca6d6b3 tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] 10.222.0.52 "GET /compute/v2.1/servers/fbea9949-6c55-4ae7-96e1-b09f1a8030f1" status: 200 len: 1314 microversion: 2.1 time: 0.163658 Aug 27 02:26:05.450126 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 249/499] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:26:05 2022] GET /compute/v2.1/servers/fbea9949-6c55-4ae7-96e1-b09f1a8030f1 => generated 1314 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:26:05.543513 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-ded2d13d-ec6f-49eb-a7ed-814f06e68c35 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Calling method '>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:26:05.544227 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-ded2d13d-ec6f-49eb-a7ed-814f06e68c35 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] [instance: 98e4cfeb-d49e-4888-a2c0-40a9e8af90d3] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:26:05.551148 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-ded2d13d-ec6f-49eb-a7ed-814f06e68c35 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:26:05.551637 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-ded2d13d-ec6f-49eb-a7ed-814f06e68c35 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:26:05.552072 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-ded2d13d-ec6f-49eb-a7ed-814f06e68c35 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:26:05.552757 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-ded2d13d-ec6f-49eb-a7ed-814f06e68c35 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:26:05.553400 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-ded2d13d-ec6f-49eb-a7ed-814f06e68c35 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:26:05.553842 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-ded2d13d-ec6f-49eb-a7ed-814f06e68c35 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:26:05.701950 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-ded2d13d-ec6f-49eb-a7ed-814f06e68c35 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd945d512ae034ec08838d0a3afde2429', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '880ac4003248441c809a76cfd1b221eb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:26:05.710793 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-ded2d13d-ec6f-49eb-a7ed-814f06e68c35 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] 10.222.0.52 "GET /compute/v2.1/servers/98e4cfeb-d49e-4888-a2c0-40a9e8af90d3" status: 200 len: 1370 microversion: 2.1 time: 0.170697 Aug 27 02:26:05.711880 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 251/500] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:26:05 2022] GET /compute/v2.1/servers/98e4cfeb-d49e-4888-a2c0-40a9e8af90d3 => generated 1370 bytes in 173 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:26:06.471717 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-cf9aef01-af12-42b3-b4ac-243ba164d6b6 tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:26:06.472423 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-cf9aef01-af12-42b3-b4ac-243ba164d6b6 tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] [instance: fbea9949-6c55-4ae7-96e1-b09f1a8030f1] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:26:06.479427 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-cf9aef01-af12-42b3-b4ac-243ba164d6b6 tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:26:06.480052 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-cf9aef01-af12-42b3-b4ac-243ba164d6b6 tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:26:06.480450 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-cf9aef01-af12-42b3-b4ac-243ba164d6b6 tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:26:06.481024 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-cf9aef01-af12-42b3-b4ac-243ba164d6b6 tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:26:06.481506 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-cf9aef01-af12-42b3-b4ac-243ba164d6b6 tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:26:06.481921 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-cf9aef01-af12-42b3-b4ac-243ba164d6b6 tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:26:06.640730 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-cf9aef01-af12-42b3-b4ac-243ba164d6b6 tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2e60cf2931f04f0e8e911adea32979dd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7cb3e3496c1a45c8942c3678c0c299e5', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:26:06.648794 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-cf9aef01-af12-42b3-b4ac-243ba164d6b6 tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] 10.222.0.52 "GET /compute/v2.1/servers/fbea9949-6c55-4ae7-96e1-b09f1a8030f1" status: 200 len: 1314 microversion: 2.1 time: 0.181252 Aug 27 02:26:06.649404 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 250/501] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:26:06 2022] GET /compute/v2.1/servers/fbea9949-6c55-4ae7-96e1-b09f1a8030f1 => generated 1314 bytes in 182 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:26:06.731960 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-28647663-6bc8-4f76-8534-30656700d80c tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Calling method '>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:26:06.732624 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-28647663-6bc8-4f76-8534-30656700d80c tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] [instance: 98e4cfeb-d49e-4888-a2c0-40a9e8af90d3] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:26:06.739272 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-28647663-6bc8-4f76-8534-30656700d80c tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:26:06.739756 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-28647663-6bc8-4f76-8534-30656700d80c tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:26:06.740252 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-28647663-6bc8-4f76-8534-30656700d80c tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:26:06.740945 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-28647663-6bc8-4f76-8534-30656700d80c tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:26:06.741434 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-28647663-6bc8-4f76-8534-30656700d80c tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:26:06.741853 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-28647663-6bc8-4f76-8534-30656700d80c tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:26:06.877306 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-28647663-6bc8-4f76-8534-30656700d80c tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd945d512ae034ec08838d0a3afde2429', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '880ac4003248441c809a76cfd1b221eb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:26:06.885489 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-28647663-6bc8-4f76-8534-30656700d80c tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] 10.222.0.52 "GET /compute/v2.1/servers/98e4cfeb-d49e-4888-a2c0-40a9e8af90d3" status: 200 len: 1370 microversion: 2.1 time: 0.157307 Aug 27 02:26:06.886058 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 252/502] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:26:06 2022] GET /compute/v2.1/servers/98e4cfeb-d49e-4888-a2c0-40a9e8af90d3 => generated 1370 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 27 02:26:07.672116 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-17631a4e-c72e-4d8a-a368-56f151f5eedd tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:26:07.672903 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-17631a4e-c72e-4d8a-a368-56f151f5eedd tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] [instance: fbea9949-6c55-4ae7-96e1-b09f1a8030f1] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:26:07.681154 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-17631a4e-c72e-4d8a-a368-56f151f5eedd tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:26:07.681647 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-17631a4e-c72e-4d8a-a368-56f151f5eedd tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:26:07.682074 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-17631a4e-c72e-4d8a-a368-56f151f5eedd tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:26:07.682759 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-17631a4e-c72e-4d8a-a368-56f151f5eedd tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:26:07.683208 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-17631a4e-c72e-4d8a-a368-56f151f5eedd tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:26:07.683622 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-17631a4e-c72e-4d8a-a368-56f151f5eedd tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:26:07.828563 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-17631a4e-c72e-4d8a-a368-56f151f5eedd tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2e60cf2931f04f0e8e911adea32979dd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7cb3e3496c1a45c8942c3678c0c299e5', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:26:07.837089 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-17631a4e-c72e-4d8a-a368-56f151f5eedd tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] 10.222.0.52 "GET /compute/v2.1/servers/fbea9949-6c55-4ae7-96e1-b09f1a8030f1" status: 200 len: 1496 microversion: 2.1 time: 0.170612 Aug 27 02:26:07.837694 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 251/503] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:26:07 2022] GET /compute/v2.1/servers/fbea9949-6c55-4ae7-96e1-b09f1a8030f1 => generated 1496 bytes in 172 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:26:07.918174 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-70633672-9808-49ea-acc1-9ed59c13397e tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Calling method '>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:26:07.918780 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-70633672-9808-49ea-acc1-9ed59c13397e tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] [instance: 98e4cfeb-d49e-4888-a2c0-40a9e8af90d3] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:26:07.925443 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-70633672-9808-49ea-acc1-9ed59c13397e tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:26:07.926236 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-70633672-9808-49ea-acc1-9ed59c13397e tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:26:07.926814 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-70633672-9808-49ea-acc1-9ed59c13397e tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:26:07.927508 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-70633672-9808-49ea-acc1-9ed59c13397e tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:26:07.928557 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-70633672-9808-49ea-acc1-9ed59c13397e tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:26:07.928685 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-70633672-9808-49ea-acc1-9ed59c13397e tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:26:08.070779 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-70633672-9808-49ea-acc1-9ed59c13397e tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd945d512ae034ec08838d0a3afde2429', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '880ac4003248441c809a76cfd1b221eb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:26:08.079962 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-70633672-9808-49ea-acc1-9ed59c13397e tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] 10.222.0.52 "GET /compute/v2.1/servers/98e4cfeb-d49e-4888-a2c0-40a9e8af90d3" status: 200 len: 1370 microversion: 2.1 time: 0.165525 Aug 27 02:26:08.080675 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 253/504] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:26:07 2022] GET /compute/v2.1/servers/98e4cfeb-d49e-4888-a2c0-40a9e8af90d3 => generated 1370 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:26:08.531968 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-a3d8cf1d-ebdc-43d1-ac94-b1f6b88b1f4c tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:26:08.532657 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-a3d8cf1d-ebdc-43d1-ac94-b1f6b88b1f4c tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] [instance: fbea9949-6c55-4ae7-96e1-b09f1a8030f1] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:26:08.541496 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-a3d8cf1d-ebdc-43d1-ac94-b1f6b88b1f4c tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:26:08.542250 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-a3d8cf1d-ebdc-43d1-ac94-b1f6b88b1f4c tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:26:08.542964 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-a3d8cf1d-ebdc-43d1-ac94-b1f6b88b1f4c tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:26:08.543969 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-a3d8cf1d-ebdc-43d1-ac94-b1f6b88b1f4c tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:26:08.544640 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-a3d8cf1d-ebdc-43d1-ac94-b1f6b88b1f4c tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:26:08.545096 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-a3d8cf1d-ebdc-43d1-ac94-b1f6b88b1f4c tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:26:08.571616 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-a3d8cf1d-ebdc-43d1-ac94-b1f6b88b1f4c tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] [instance: fbea9949-6c55-4ae7-96e1-b09f1a8030f1] Going to try to terminate instance {{(pid=120863) delete /opt/stack/nova/nova/compute/api.py:2719}} Aug 27 02:26:08.681929 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-a3d8cf1d-ebdc-43d1-ac94-b1f6b88b1f4c tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] 10.222.0.52 "DELETE /compute/v2.1/servers/fbea9949-6c55-4ae7-96e1-b09f1a8030f1" status: 204 len: 0 microversion: 2.1 time: 0.155915 Aug 27 02:26:08.682737 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 252/505] 10.222.0.52 () {58 vars in 1239 bytes} [Sat Aug 27 02:26:08 2022] DELETE /compute/v2.1/servers/fbea9949-6c55-4ae7-96e1-b09f1a8030f1 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Aug 27 02:26:08.695363 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-49a7a3fd-93ea-466c-92c9-a690fb749f59 tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Calling method '>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:26:08.695899 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-49a7a3fd-93ea-466c-92c9-a690fb749f59 tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] [instance: fbea9949-6c55-4ae7-96e1-b09f1a8030f1] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:26:08.702908 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-49a7a3fd-93ea-466c-92c9-a690fb749f59 tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:26:08.703411 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-49a7a3fd-93ea-466c-92c9-a690fb749f59 tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:26:08.703849 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-49a7a3fd-93ea-466c-92c9-a690fb749f59 tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:26:08.704534 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-49a7a3fd-93ea-466c-92c9-a690fb749f59 tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:26:08.704981 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-49a7a3fd-93ea-466c-92c9-a690fb749f59 tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:26:08.705423 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-49a7a3fd-93ea-466c-92c9-a690fb749f59 tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:26:08.854787 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-49a7a3fd-93ea-466c-92c9-a690fb749f59 tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2e60cf2931f04f0e8e911adea32979dd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7cb3e3496c1a45c8942c3678c0c299e5', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:26:08.864899 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-49a7a3fd-93ea-466c-92c9-a690fb749f59 tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] 10.222.0.52 "GET /compute/v2.1/servers/fbea9949-6c55-4ae7-96e1-b09f1a8030f1" status: 200 len: 1502 microversion: 2.1 time: 0.173121 Aug 27 02:26:08.865146 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 254/506] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:26:08 2022] GET /compute/v2.1/servers/fbea9949-6c55-4ae7-96e1-b09f1a8030f1 => generated 1502 bytes in 175 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:26:08.910666 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [req-93ef7196-66ee-4425-9aa5-6665aee02947 req-a761e992-6a39-4b74-8a73-f4eec62c465c service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "98e4cfeb-d49e-4888-a2c0-40a9e8af90d3", "name": "network-vif-plugged", "status": "completed", "tag": "bf498aa0-7dfa-466b-b4f0-3a5d8f8703e2"}]} {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 27 02:26:08.920381 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [req-93ef7196-66ee-4425-9aa5-6665aee02947 req-a761e992-6a39-4b74-8a73-f4eec62c465c service nova] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:26:08.920880 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [req-93ef7196-66ee-4425-9aa5-6665aee02947 req-a761e992-6a39-4b74-8a73-f4eec62c465c service nova] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:26:08.921430 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [req-93ef7196-66ee-4425-9aa5-6665aee02947 req-a761e992-6a39-4b74-8a73-f4eec62c465c service nova] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:26:08.933163 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.compute.server_external_events [req-93ef7196-66ee-4425-9aa5-6665aee02947 req-a761e992-6a39-4b74-8a73-f4eec62c465c service nova] Creating event network-vif-plugged:bf498aa0-7dfa-466b-b4f0-3a5d8f8703e2 for instance 98e4cfeb-d49e-4888-a2c0-40a9e8af90d3 on hv-cinder-84630 Aug 27 02:26:08.937794 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [req-93ef7196-66ee-4425-9aa5-6665aee02947 req-a761e992-6a39-4b74-8a73-f4eec62c465c service nova] 10.222.0.52 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.032450 Aug 27 02:26:08.938388 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 253/507] 10.222.0.52 () {64 vars in 1317 bytes} [Sat Aug 27 02:26:08 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 34 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 27 02:26:09.103149 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-433a8cca-5347-4ae9-99d5-498e4d2f7670 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Calling method '>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:26:09.103808 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-433a8cca-5347-4ae9-99d5-498e4d2f7670 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] [instance: 98e4cfeb-d49e-4888-a2c0-40a9e8af90d3] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:26:09.110102 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-433a8cca-5347-4ae9-99d5-498e4d2f7670 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:26:09.110588 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-433a8cca-5347-4ae9-99d5-498e4d2f7670 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:26:09.111017 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-433a8cca-5347-4ae9-99d5-498e4d2f7670 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:26:09.111710 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-433a8cca-5347-4ae9-99d5-498e4d2f7670 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:26:09.112162 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-433a8cca-5347-4ae9-99d5-498e4d2f7670 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:26:09.112581 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-433a8cca-5347-4ae9-99d5-498e4d2f7670 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:26:09.255186 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-433a8cca-5347-4ae9-99d5-498e4d2f7670 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd945d512ae034ec08838d0a3afde2429', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '880ac4003248441c809a76cfd1b221eb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:26:09.263387 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-433a8cca-5347-4ae9-99d5-498e4d2f7670 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] 10.222.0.52 "GET /compute/v2.1/servers/98e4cfeb-d49e-4888-a2c0-40a9e8af90d3" status: 200 len: 1370 microversion: 2.1 time: 0.164606 Aug 27 02:26:09.263965 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 255/508] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:26:09 2022] GET /compute/v2.1/servers/98e4cfeb-d49e-4888-a2c0-40a9e8af90d3 => generated 1370 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:26:09.886925 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-f7a89a78-9094-4c7e-a4a6-3e524082e7b3 tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:26:09.887848 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-f7a89a78-9094-4c7e-a4a6-3e524082e7b3 tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] [instance: fbea9949-6c55-4ae7-96e1-b09f1a8030f1] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:26:09.895976 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-f7a89a78-9094-4c7e-a4a6-3e524082e7b3 tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:26:09.896679 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-f7a89a78-9094-4c7e-a4a6-3e524082e7b3 tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:26:09.897119 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-f7a89a78-9094-4c7e-a4a6-3e524082e7b3 tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:26:09.898219 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-f7a89a78-9094-4c7e-a4a6-3e524082e7b3 tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:26:09.898835 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-f7a89a78-9094-4c7e-a4a6-3e524082e7b3 tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:26:09.899459 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-f7a89a78-9094-4c7e-a4a6-3e524082e7b3 tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:26:10.046502 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-f7a89a78-9094-4c7e-a4a6-3e524082e7b3 tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2e60cf2931f04f0e8e911adea32979dd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7cb3e3496c1a45c8942c3678c0c299e5', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:26:10.051752 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG dbcounter [-] [120863] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=120863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 27 02:26:10.056679 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-f7a89a78-9094-4c7e-a4a6-3e524082e7b3 tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] 10.222.0.52 "GET /compute/v2.1/servers/fbea9949-6c55-4ae7-96e1-b09f1a8030f1" status: 200 len: 1502 microversion: 2.1 time: 0.174479 Aug 27 02:26:10.057276 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 254/509] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:26:09 2022] GET /compute/v2.1/servers/fbea9949-6c55-4ae7-96e1-b09f1a8030f1 => generated 1502 bytes in 176 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:26:10.286114 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-7ee802c2-0315-4467-b5aa-a0bb72a8982f tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Calling method '>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:26:10.286856 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-7ee802c2-0315-4467-b5aa-a0bb72a8982f tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] [instance: 98e4cfeb-d49e-4888-a2c0-40a9e8af90d3] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:26:10.292865 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-7ee802c2-0315-4467-b5aa-a0bb72a8982f tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:26:10.293520 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-7ee802c2-0315-4467-b5aa-a0bb72a8982f tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:26:10.293830 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-7ee802c2-0315-4467-b5aa-a0bb72a8982f tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:26:10.294761 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-7ee802c2-0315-4467-b5aa-a0bb72a8982f tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:26:10.295189 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-7ee802c2-0315-4467-b5aa-a0bb72a8982f tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:26:10.295640 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-7ee802c2-0315-4467-b5aa-a0bb72a8982f tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:26:10.478876 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-7ee802c2-0315-4467-b5aa-a0bb72a8982f tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd945d512ae034ec08838d0a3afde2429', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '880ac4003248441c809a76cfd1b221eb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:26:10.488251 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-7ee802c2-0315-4467-b5aa-a0bb72a8982f tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] 10.222.0.52 "GET /compute/v2.1/servers/98e4cfeb-d49e-4888-a2c0-40a9e8af90d3" status: 200 len: 1370 microversion: 2.1 time: 0.207227 Aug 27 02:26:10.489576 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 256/510] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:26:10 2022] GET /compute/v2.1/servers/98e4cfeb-d49e-4888-a2c0-40a9e8af90d3 => generated 1370 bytes in 209 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:26:11.080940 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-32c66904-d36d-4142-8e37-0e990dbed84b tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:26:11.081647 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-32c66904-d36d-4142-8e37-0e990dbed84b tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] [instance: fbea9949-6c55-4ae7-96e1-b09f1a8030f1] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:26:11.089656 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-32c66904-d36d-4142-8e37-0e990dbed84b tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:26:11.090143 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-32c66904-d36d-4142-8e37-0e990dbed84b tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:26:11.090582 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-32c66904-d36d-4142-8e37-0e990dbed84b tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:26:11.091671 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-32c66904-d36d-4142-8e37-0e990dbed84b tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:26:11.092186 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-32c66904-d36d-4142-8e37-0e990dbed84b tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:26:11.092870 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-32c66904-d36d-4142-8e37-0e990dbed84b tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:26:11.237645 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-32c66904-d36d-4142-8e37-0e990dbed84b tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2e60cf2931f04f0e8e911adea32979dd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7cb3e3496c1a45c8942c3678c0c299e5', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:26:11.246102 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-32c66904-d36d-4142-8e37-0e990dbed84b tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] 10.222.0.52 "GET /compute/v2.1/servers/fbea9949-6c55-4ae7-96e1-b09f1a8030f1" status: 200 len: 1502 microversion: 2.1 time: 0.169861 Aug 27 02:26:11.246676 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 255/511] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:26:11 2022] GET /compute/v2.1/servers/fbea9949-6c55-4ae7-96e1-b09f1a8030f1 => generated 1502 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:26:11.512185 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-e89dca2c-35a3-4c4c-89f0-b97c5533028e tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Calling method '>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:26:11.512965 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-e89dca2c-35a3-4c4c-89f0-b97c5533028e tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] [instance: 98e4cfeb-d49e-4888-a2c0-40a9e8af90d3] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:26:11.520063 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-e89dca2c-35a3-4c4c-89f0-b97c5533028e tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:26:11.520502 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-e89dca2c-35a3-4c4c-89f0-b97c5533028e tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:26:11.521003 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-e89dca2c-35a3-4c4c-89f0-b97c5533028e tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:26:11.521791 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-e89dca2c-35a3-4c4c-89f0-b97c5533028e tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:26:11.522285 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-e89dca2c-35a3-4c4c-89f0-b97c5533028e tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:26:11.522793 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-e89dca2c-35a3-4c4c-89f0-b97c5533028e tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:26:11.666209 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-e89dca2c-35a3-4c4c-89f0-b97c5533028e tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd945d512ae034ec08838d0a3afde2429', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '880ac4003248441c809a76cfd1b221eb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:26:11.669768 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG dbcounter [-] [120864] Writing DB stats nova_cell1:SELECT=100 {{(pid=120864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 27 02:26:11.674825 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-e89dca2c-35a3-4c4c-89f0-b97c5533028e tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] 10.222.0.52 "GET /compute/v2.1/servers/98e4cfeb-d49e-4888-a2c0-40a9e8af90d3" status: 200 len: 1370 microversion: 2.1 time: 0.167651 Aug 27 02:26:11.675402 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 257/512] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:26:11 2022] GET /compute/v2.1/servers/98e4cfeb-d49e-4888-a2c0-40a9e8af90d3 => generated 1370 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:26:12.267873 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-d6530798-001d-4af4-af49-a1785cc293f1 tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:26:12.268432 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-d6530798-001d-4af4-af49-a1785cc293f1 tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] [instance: fbea9949-6c55-4ae7-96e1-b09f1a8030f1] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:26:12.275254 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-d6530798-001d-4af4-af49-a1785cc293f1 tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:26:12.275731 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-d6530798-001d-4af4-af49-a1785cc293f1 tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:26:12.276156 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-d6530798-001d-4af4-af49-a1785cc293f1 tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:26:12.276841 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-d6530798-001d-4af4-af49-a1785cc293f1 tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:26:12.277319 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-d6530798-001d-4af4-af49-a1785cc293f1 tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:26:12.277738 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-d6530798-001d-4af4-af49-a1785cc293f1 tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:26:12.416523 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-d6530798-001d-4af4-af49-a1785cc293f1 tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2e60cf2931f04f0e8e911adea32979dd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7cb3e3496c1a45c8942c3678c0c299e5', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:26:12.424301 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-d6530798-001d-4af4-af49-a1785cc293f1 tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] 10.222.0.52 "GET /compute/v2.1/servers/fbea9949-6c55-4ae7-96e1-b09f1a8030f1" status: 200 len: 1502 microversion: 2.1 time: 0.160467 Aug 27 02:26:12.424846 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 256/513] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:26:12 2022] GET /compute/v2.1/servers/fbea9949-6c55-4ae7-96e1-b09f1a8030f1 => generated 1502 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:26:12.696292 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-121f58f5-a91f-4902-8db6-03ec3bcad26e tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Calling method '>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:26:12.696801 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [req-8e1dfa11-e4f9-451d-8a57-6cdd9da81f03 req-7b262c7a-dbcb-4b89-80a6-b3b3b8a669fb service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "fbea9949-6c55-4ae7-96e1-b09f1a8030f1", "name": "network-vif-deleted", "tag": "0d4a34a6-1648-44be-9eb9-79840124e8e3"}]} {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 27 02:26:12.696918 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-121f58f5-a91f-4902-8db6-03ec3bcad26e tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] [instance: 98e4cfeb-d49e-4888-a2c0-40a9e8af90d3] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:26:12.703816 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-121f58f5-a91f-4902-8db6-03ec3bcad26e tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:26:12.704217 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-121f58f5-a91f-4902-8db6-03ec3bcad26e tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:26:12.704680 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-121f58f5-a91f-4902-8db6-03ec3bcad26e tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:26:12.705518 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-121f58f5-a91f-4902-8db6-03ec3bcad26e tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:26:12.706159 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-121f58f5-a91f-4902-8db6-03ec3bcad26e tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:26:12.706574 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-121f58f5-a91f-4902-8db6-03ec3bcad26e tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:26:12.707470 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [req-8e1dfa11-e4f9-451d-8a57-6cdd9da81f03 req-7b262c7a-dbcb-4b89-80a6-b3b3b8a669fb service nova] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:26:12.708052 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [req-8e1dfa11-e4f9-451d-8a57-6cdd9da81f03 req-7b262c7a-dbcb-4b89-80a6-b3b3b8a669fb service nova] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:26:12.708578 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [req-8e1dfa11-e4f9-451d-8a57-6cdd9da81f03 req-7b262c7a-dbcb-4b89-80a6-b3b3b8a669fb service nova] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:26:12.731273 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.compute.server_external_events [req-8e1dfa11-e4f9-451d-8a57-6cdd9da81f03 req-7b262c7a-dbcb-4b89-80a6-b3b3b8a669fb service nova] Creating event network-vif-deleted:0d4a34a6-1648-44be-9eb9-79840124e8e3 for instance fbea9949-6c55-4ae7-96e1-b09f1a8030f1 on hv-cinder-84630 Aug 27 02:26:12.736928 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [req-8e1dfa11-e4f9-451d-8a57-6cdd9da81f03 req-7b262c7a-dbcb-4b89-80a6-b3b3b8a669fb service nova] 10.222.0.52 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.044356 Aug 27 02:26:12.737732 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 257/514] 10.222.0.52 () {64 vars in 1317 bytes} [Sat Aug 27 02:26:12 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Aug 27 02:26:12.846735 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-121f58f5-a91f-4902-8db6-03ec3bcad26e tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd945d512ae034ec08838d0a3afde2429', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '880ac4003248441c809a76cfd1b221eb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:26:12.855011 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-121f58f5-a91f-4902-8db6-03ec3bcad26e tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] 10.222.0.52 "GET /compute/v2.1/servers/98e4cfeb-d49e-4888-a2c0-40a9e8af90d3" status: 200 len: 1370 microversion: 2.1 time: 0.162459 Aug 27 02:26:12.855559 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 258/515] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:26:12 2022] GET /compute/v2.1/servers/98e4cfeb-d49e-4888-a2c0-40a9e8af90d3 => generated 1370 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:26:13.449007 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-df7135f8-0083-4855-be95-79c3681d9025 tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:26:13.449611 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-df7135f8-0083-4855-be95-79c3681d9025 tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] [instance: fbea9949-6c55-4ae7-96e1-b09f1a8030f1] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:26:13.457218 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-df7135f8-0083-4855-be95-79c3681d9025 tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:26:13.457707 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-df7135f8-0083-4855-be95-79c3681d9025 tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:26:13.458139 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-df7135f8-0083-4855-be95-79c3681d9025 tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:26:13.458836 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-df7135f8-0083-4855-be95-79c3681d9025 tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:26:13.459288 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-df7135f8-0083-4855-be95-79c3681d9025 tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:26:13.459706 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-df7135f8-0083-4855-be95-79c3681d9025 tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:26:13.535957 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-df7135f8-0083-4855-be95-79c3681d9025 tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2e60cf2931f04f0e8e911adea32979dd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7cb3e3496c1a45c8942c3678c0c299e5', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:26:13.543888 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-df7135f8-0083-4855-be95-79c3681d9025 tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] 10.222.0.52 "GET /compute/v2.1/servers/fbea9949-6c55-4ae7-96e1-b09f1a8030f1" status: 200 len: 1460 microversion: 2.1 time: 0.101036 Aug 27 02:26:13.544451 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 258/516] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:26:13 2022] GET /compute/v2.1/servers/fbea9949-6c55-4ae7-96e1-b09f1a8030f1 => generated 1460 bytes in 103 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 27 02:26:13.876087 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-62eef7ca-67db-4737-98fe-21c3684fc8a3 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Calling method '>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:26:13.876706 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-62eef7ca-67db-4737-98fe-21c3684fc8a3 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] [instance: 98e4cfeb-d49e-4888-a2c0-40a9e8af90d3] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:26:13.883685 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-62eef7ca-67db-4737-98fe-21c3684fc8a3 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:26:13.884313 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-62eef7ca-67db-4737-98fe-21c3684fc8a3 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:26:13.884735 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-62eef7ca-67db-4737-98fe-21c3684fc8a3 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:26:13.885451 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-62eef7ca-67db-4737-98fe-21c3684fc8a3 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:26:13.885893 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-62eef7ca-67db-4737-98fe-21c3684fc8a3 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:26:13.886307 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-62eef7ca-67db-4737-98fe-21c3684fc8a3 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:26:14.025142 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-62eef7ca-67db-4737-98fe-21c3684fc8a3 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd945d512ae034ec08838d0a3afde2429', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '880ac4003248441c809a76cfd1b221eb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:26:14.033154 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-62eef7ca-67db-4737-98fe-21c3684fc8a3 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] 10.222.0.52 "GET /compute/v2.1/servers/98e4cfeb-d49e-4888-a2c0-40a9e8af90d3" status: 200 len: 1564 microversion: 2.1 time: 0.161039 Aug 27 02:26:14.033706 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 259/517] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:26:13 2022] GET /compute/v2.1/servers/98e4cfeb-d49e-4888-a2c0-40a9e8af90d3 => generated 1564 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:26:14.054292 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-06436ccb-f950-4843-8f9c-c07846cbc297 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:26:14.054777 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-06436ccb-f950-4843-8f9c-c07846cbc297 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] [instance: 98e4cfeb-d49e-4888-a2c0-40a9e8af90d3] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:26:14.060951 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-06436ccb-f950-4843-8f9c-c07846cbc297 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:26:14.061623 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-06436ccb-f950-4843-8f9c-c07846cbc297 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:26:14.062053 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-06436ccb-f950-4843-8f9c-c07846cbc297 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:26:14.062740 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-06436ccb-f950-4843-8f9c-c07846cbc297 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:26:14.063190 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-06436ccb-f950-4843-8f9c-c07846cbc297 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:26:14.063604 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-06436ccb-f950-4843-8f9c-c07846cbc297 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:26:14.144578 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-06436ccb-f950-4843-8f9c-c07846cbc297 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] 10.222.0.52 "GET /compute/v2.1/servers/98e4cfeb-d49e-4888-a2c0-40a9e8af90d3/os-interface" status: 200 len: 285 microversion: 2.1 time: 0.094206 Aug 27 02:26:14.145405 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 259/518] 10.222.0.52 () {58 vars in 1275 bytes} [Sat Aug 27 02:26:14 2022] GET /compute/v2.1/servers/98e4cfeb-d49e-4888-a2c0-40a9e8af90d3/os-interface => generated 285 bytes in 96 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 27 02:26:14.567068 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-8703339c-bc4b-4915-8cc8-ad60b77920e5 tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Calling method '>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:26:14.567874 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-8703339c-bc4b-4915-8cc8-ad60b77920e5 tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] [instance: fbea9949-6c55-4ae7-96e1-b09f1a8030f1] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:26:14.575030 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-8703339c-bc4b-4915-8cc8-ad60b77920e5 tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:26:14.575511 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-8703339c-bc4b-4915-8cc8-ad60b77920e5 tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:26:14.577220 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-8703339c-bc4b-4915-8cc8-ad60b77920e5 tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:26:14.577220 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-8703339c-bc4b-4915-8cc8-ad60b77920e5 tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:26:14.577574 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-8703339c-bc4b-4915-8cc8-ad60b77920e5 tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:26:14.578002 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-8703339c-bc4b-4915-8cc8-ad60b77920e5 tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:26:14.593223 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.wsgi [None req-8703339c-bc4b-4915-8cc8-ad60b77920e5 tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] HTTP exception thrown: Instance fbea9949-6c55-4ae7-96e1-b09f1a8030f1 could not be found. Aug 27 02:26:14.594875 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-8703339c-bc4b-4915-8cc8-ad60b77920e5 tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Request method failure captured: Aug 27 02:26:14.594875 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: request: GET /compute/v2.1/servers/fbea9949-6c55-4ae7-96e1-b09f1a8030f1 HTTP/1.1 Aug 27 02:26:14.594875 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: Accept: application/json Aug 27 02:26:14.594875 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: Accept-Encoding: identity Aug 27 02:26:14.594875 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: Connection: close Aug 27 02:26:14.594875 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: Content-Length: 0 Aug 27 02:26:14.594875 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: Content-Type: application/json Aug 27 02:26:14.594875 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: Host: 10.222.0.52 Aug 27 02:26:14.594875 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: Openstack-System-Scope: None Aug 27 02:26:14.594875 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: User-Agent: python-urllib3/1.26.11 Aug 27 02:26:14.594875 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-Auth-Token: gAAAAABjCYB0MyJqOfE5kKRO8uoWsCg0klSjiu7lxuQEUldbXVIPAZZ5yV6_hhSVHubtYc4X9p9JyWfm8mZ1gFcYpJGdJ8Ek7ubkjfN8IvIv2cpKCCIp1zSQKzKaTdPuvOUqwCXR64qVlUgO_Kp5x1BMLZilD49VGKqBjblPsc1kPAfNx47Clds Aug 27 02:26:14.594875 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-Domain-Id: None Aug 27 02:26:14.594875 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-Domain-Name: None Aug 27 02:26:14.594875 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-Identity-Status: Confirmed Aug 27 02:26:14.594875 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-Is-Admin-Project: True Aug 27 02:26:14.594875 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-Project-Domain-Id: default Aug 27 02:26:14.594875 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-Project-Domain-Name: Default Aug 27 02:26:14.594875 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-Project-Id: 7cb3e3496c1a45c8942c3678c0c299e5 Aug 27 02:26:14.594875 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-Project-Name: tempest-VolumesActionsTest-477223312 Aug 27 02:26:14.594875 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-Role: reader,member Aug 27 02:26:14.594875 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-Roles: reader,member Aug 27 02:26:14.596858 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-Service-Catalog: [{"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.52:8080", "publicURL": "http://10.222.0.52:8080/v1/AUTH_7cb3e3496c1a45c8942c3678c0c299e5"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52:9696/networking"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/compute/v2/7cb3e3496c1a45c8942c3678c0c299e5"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/image"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/volume/v3/7cb3e3496c1a45c8942c3678c0c299e5"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/identity"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/volume/v3/7cb3e3496c1a45c8942c3678c0c299e5"}]}] Aug 27 02:26:14.596858 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-Tenant: tempest-VolumesActionsTest-477223312 Aug 27 02:26:14.596858 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-Tenant-Id: 7cb3e3496c1a45c8942c3678c0c299e5 Aug 27 02:26:14.596858 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-Tenant-Name: tempest-VolumesActionsTest-477223312 Aug 27 02:26:14.596858 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-User: tempest-VolumesActionsTest-477223312-project Aug 27 02:26:14.596858 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-User-Domain-Id: default Aug 27 02:26:14.596858 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-User-Domain-Name: Default Aug 27 02:26:14.596858 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-User-Id: 2e60cf2931f04f0e8e911adea32979dd Aug 27 02:26:14.596858 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-User-Name: tempest-VolumesActionsTest-477223312-project Aug 27 02:26:14.596858 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: method: > Aug 27 02:26:14.596858 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: exception: Instance fbea9949-6c55-4ae7-96e1-b09f1a8030f1 could not be found. Aug 27 02:26:14.596858 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: action_args: {'id': 'fbea9949-6c55-4ae7-96e1-b09f1a8030f1'} Aug 27 02:26:14.597671 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Aug 27 02:26:14.597671 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 27 02:26:14.597671 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Aug 27 02:26:14.597671 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Aug 27 02:26:14.597671 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Aug 27 02:26:14.597671 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Aug 27 02:26:14.597671 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Aug 27 02:26:14.597671 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Aug 27 02:26:14.597671 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Aug 27 02:26:14.597671 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Aug 27 02:26:14.597671 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance fbea9949-6c55-4ae7-96e1-b09f1a8030f1 could not be found. Aug 27 02:26:14.597671 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi Aug 27 02:26:14.597671 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 27 02:26:14.597671 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi Aug 27 02:26:14.597671 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 27 02:26:14.597671 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 27 02:26:14.597671 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 27 02:26:14.597671 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Aug 27 02:26:14.597671 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Aug 27 02:26:14.597671 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Aug 27 02:26:14.598874 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Aug 27 02:26:14.598874 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Aug 27 02:26:14.598874 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi instance = self._get_server( Aug 27 02:26:14.598874 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Aug 27 02:26:14.598874 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Aug 27 02:26:14.598874 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Aug 27 02:26:14.598874 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Aug 27 02:26:14.598874 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance fbea9949-6c55-4ae7-96e1-b09f1a8030f1 could not be found. Aug 27 02:26:14.598874 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi Aug 27 02:26:14.598874 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 27 02:26:14.598874 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi Aug 27 02:26:14.598874 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 27 02:26:14.598874 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 27 02:26:14.598874 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 27 02:26:14.598874 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Aug 27 02:26:14.598874 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Aug 27 02:26:14.598874 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance fbea9949-6c55-4ae7-96e1-b09f1a8030f1 could not be found. Aug 27 02:26:14.598874 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi Aug 27 02:26:14.599938 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-8703339c-bc4b-4915-8cc8-ad60b77920e5 tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] Returning 404 to user: Instance fbea9949-6c55-4ae7-96e1-b09f1a8030f1 could not be found. {{(pid=120864) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Aug 27 02:26:14.599938 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-8703339c-bc4b-4915-8cc8-ad60b77920e5 tempest-VolumesActionsTest-477223312 tempest-VolumesActionsTest-477223312-project] 10.222.0.52 "GET /compute/v2.1/servers/fbea9949-6c55-4ae7-96e1-b09f1a8030f1" status: 404 len: 111 microversion: 2.1 time: 0.033743 Aug 27 02:26:14.599938 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 260/519] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:26:14 2022] GET /compute/v2.1/servers/fbea9949-6c55-4ae7-96e1-b09f1a8030f1 => generated 111 bytes in 35 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Aug 27 02:26:15.434698 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [req-49c305ca-a770-4d56-80e6-1bdecfcdfbd5 req-ac0aef9c-56dd-4141-8cbd-c1e9d56a7ead service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "98e4cfeb-d49e-4888-a2c0-40a9e8af90d3", "tag": "bf498aa0-7dfa-466b-b4f0-3a5d8f8703e2"}]} {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 27 02:26:15.444917 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [req-49c305ca-a770-4d56-80e6-1bdecfcdfbd5 req-ac0aef9c-56dd-4141-8cbd-c1e9d56a7ead service nova] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:26:15.445507 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [req-49c305ca-a770-4d56-80e6-1bdecfcdfbd5 req-ac0aef9c-56dd-4141-8cbd-c1e9d56a7ead service nova] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:26:15.445996 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [req-49c305ca-a770-4d56-80e6-1bdecfcdfbd5 req-ac0aef9c-56dd-4141-8cbd-c1e9d56a7ead service nova] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:26:15.457594 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.compute.server_external_events [req-49c305ca-a770-4d56-80e6-1bdecfcdfbd5 req-ac0aef9c-56dd-4141-8cbd-c1e9d56a7ead service nova] Creating event network-changed:bf498aa0-7dfa-466b-b4f0-3a5d8f8703e2 for instance 98e4cfeb-d49e-4888-a2c0-40a9e8af90d3 on hv-cinder-84630 Aug 27 02:26:15.462761 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [req-49c305ca-a770-4d56-80e6-1bdecfcdfbd5 req-ac0aef9c-56dd-4141-8cbd-c1e9d56a7ead service nova] 10.222.0.52 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.033915 Aug 27 02:26:15.463776 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 260/520] 10.222.0.52 () {64 vars in 1317 bytes} [Sat Aug 27 02:26:15 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 36 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 27 02:26:30.641379 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-ebc5d214-d658-4b9d-8a6e-302e1b6610de tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Calling method '>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:26:30.642130 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-ebc5d214-d658-4b9d-8a6e-302e1b6610de tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] [instance: 2ad09ab6-ef1f-47a2-bf44-6ae046cc8645] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:26:30.644099 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG dbcounter [-] [120864] Writing DB stats nova_api:INSERT=3,nova_api:SELECT=86 {{(pid=120864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 27 02:26:30.644801 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG dbcounter [-] [120864] Writing DB stats nova_cell1:SELECT=14 {{(pid=120864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 27 02:26:30.650601 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-ebc5d214-d658-4b9d-8a6e-302e1b6610de tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:26:30.651090 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-ebc5d214-d658-4b9d-8a6e-302e1b6610de tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:26:30.651512 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-ebc5d214-d658-4b9d-8a6e-302e1b6610de tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:26:30.652207 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-ebc5d214-d658-4b9d-8a6e-302e1b6610de tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:26:30.652656 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-ebc5d214-d658-4b9d-8a6e-302e1b6610de tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:26:30.653071 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-ebc5d214-d658-4b9d-8a6e-302e1b6610de tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:26:30.680810 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-ebc5d214-d658-4b9d-8a6e-302e1b6610de tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] 10.222.0.52 "GET /compute/v2.1/servers/2ad09ab6-ef1f-47a2-bf44-6ae046cc8645/ips" status: 200 len: 148 microversion: 2.1 time: 0.042990 Aug 27 02:26:30.685197 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 261/521] 10.222.0.52 () {58 vars in 1248 bytes} [Sat Aug 27 02:26:30 2022] GET /compute/v2.1/servers/2ad09ab6-ef1f-47a2-bf44-6ae046cc8645/ips => generated 148 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 27 02:26:33.302152 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-3deb1b6f-4914-421b-9e59-c724cd6268e8 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "58ee6df8-0ef2-453b-82ec-e4643432a653"}} {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 27 02:26:33.307440 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-3deb1b6f-4914-421b-9e59-c724cd6268e8 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] [instance: 2ad09ab6-ef1f-47a2-bf44-6ae046cc8645] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:26:33.312624 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG dbcounter [-] [120863] Writing DB stats nova_api:SELECT=68,nova_api:UPDATE=1 {{(pid=120863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 27 02:26:33.314493 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG dbcounter [-] [120863] Writing DB stats nova_cell1:SELECT=26 {{(pid=120863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 27 02:26:33.323920 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-3deb1b6f-4914-421b-9e59-c724cd6268e8 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:26:33.324697 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-3deb1b6f-4914-421b-9e59-c724cd6268e8 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:26:33.325147 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-3deb1b6f-4914-421b-9e59-c724cd6268e8 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:26:33.326221 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-3deb1b6f-4914-421b-9e59-c724cd6268e8 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:26:33.327360 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-3deb1b6f-4914-421b-9e59-c724cd6268e8 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:26:33.327945 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-3deb1b6f-4914-421b-9e59-c724cd6268e8 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:26:33.727984 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-3deb1b6f-4914-421b-9e59-c724cd6268e8 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] 10.222.0.52 "POST /compute/v2.1/servers/2ad09ab6-ef1f-47a2-bf44-6ae046cc8645/os-volume_attachments" status: 200 len: 194 microversion: 2.1 time: 0.431965 Aug 27 02:26:33.730196 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 261/522] 10.222.0.52 () {60 vars in 1323 bytes} [Sat Aug 27 02:26:33 2022] POST /compute/v2.1/servers/2ad09ab6-ef1f-47a2-bf44-6ae046cc8645/os-volume_attachments => generated 194 bytes in 436 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 27 02:26:37.272547 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-0d91b59e-40dc-40d2-aac6-a09f046ca274 tempest-VolumesSnapshotTestJSON-1750456804 tempest-VolumesSnapshotTestJSON-1750456804-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "e3c6fdb0-e4d8-4f19-b6f8-e585b9ba482c", "device": "/dev/sdb"}} {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 27 02:26:37.275450 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-0d91b59e-40dc-40d2-aac6-a09f046ca274 tempest-VolumesSnapshotTestJSON-1750456804 tempest-VolumesSnapshotTestJSON-1750456804-project] [instance: c31c26a8-fe5a-40fb-91a1-7b5251fd0daa] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:26:37.282388 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-0d91b59e-40dc-40d2-aac6-a09f046ca274 tempest-VolumesSnapshotTestJSON-1750456804 tempest-VolumesSnapshotTestJSON-1750456804-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:26:37.283249 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-0d91b59e-40dc-40d2-aac6-a09f046ca274 tempest-VolumesSnapshotTestJSON-1750456804 tempest-VolumesSnapshotTestJSON-1750456804-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:26:37.283867 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-0d91b59e-40dc-40d2-aac6-a09f046ca274 tempest-VolumesSnapshotTestJSON-1750456804 tempest-VolumesSnapshotTestJSON-1750456804-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:26:37.284742 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-0d91b59e-40dc-40d2-aac6-a09f046ca274 tempest-VolumesSnapshotTestJSON-1750456804 tempest-VolumesSnapshotTestJSON-1750456804-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:26:37.285293 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-0d91b59e-40dc-40d2-aac6-a09f046ca274 tempest-VolumesSnapshotTestJSON-1750456804 tempest-VolumesSnapshotTestJSON-1750456804-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:26:37.286159 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-0d91b59e-40dc-40d2-aac6-a09f046ca274 tempest-VolumesSnapshotTestJSON-1750456804 tempest-VolumesSnapshotTestJSON-1750456804-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:26:37.659190 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-0d91b59e-40dc-40d2-aac6-a09f046ca274 tempest-VolumesSnapshotTestJSON-1750456804 tempest-VolumesSnapshotTestJSON-1750456804-project] 10.222.0.52 "POST /compute/v2.1/servers/c31c26a8-fe5a-40fb-91a1-7b5251fd0daa/os-volume_attachments" status: 200 len: 194 microversion: 2.1 time: 0.388768 Aug 27 02:26:37.661899 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 262/523] 10.222.0.52 () {60 vars in 1323 bytes} [Sat Aug 27 02:26:37 2022] POST /compute/v2.1/servers/c31c26a8-fe5a-40fb-91a1-7b5251fd0daa/os-volume_attachments => generated 194 bytes in 394 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 27 02:26:42.491112 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-d8f64f9b-9a72-4526-9113-4f973c676d7a tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Action: 'create', calling method: .version_select at 0x7f6b1b3b89d0>, body: {"keypair": {"name": "tempest-keypair-383499095", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBBEG8ERGhOrID97nRQkTnESp6woVWROm3qJ8L8p9nSlbufodwQzhIKVPTzSKah81RKY/772hWDbOA1ZnDcQlzAMJV5CB+v8F1uFfnZIUmGJORw7hZjAqxVBfJ+nFHfYrhA=="}} {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 27 02:26:42.508181 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.quota [None req-d8f64f9b-9a72-4526-9113-4f973c676d7a tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Getting quotas for project 5d5ec8450c0941d18d63558baab54737. Resources: {'key_pairs'} {{(pid=120863) _get_quotas /opt/stack/nova/nova/quota.py:393}} Aug 27 02:26:42.512279 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.quota [None req-d8f64f9b-9a72-4526-9113-4f973c676d7a tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Getting quotas for user 26a86351583d49d090b3db09f127d49f and project 5d5ec8450c0941d18d63558baab54737. Resources: {'key_pairs'} {{(pid=120863) _get_quotas /opt/stack/nova/nova/quota.py:383}} Aug 27 02:26:42.538868 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-d8f64f9b-9a72-4526-9113-4f973c676d7a tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] 10.222.0.52 "POST /compute/v2.1/os-keypairs" status: 200 len: 385 microversion: 2.1 time: 0.052603 Aug 27 02:26:42.539505 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 262/524] 10.222.0.52 () {60 vars in 1159 bytes} [Sat Aug 27 02:26:42 2022] POST /compute/v2.1/os-keypairs => generated 385 bytes in 54 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 27 02:26:44.173950 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-010a9ff7-008c-4e6a-a907-489f660cef57 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-AttachVolumeNegativeTest-server-1459696288", "imageRef": "9fc30b36-b827-46f9-892d-30e808bc2b8a", "flavorRef": "452", "adminPass": "***", "networks": [{"uuid": "134184f4-d632-4173-b77c-1566bf4eb0fb"}], "security_groups": [{"name": "tempest-securitygroup--921117983"}], "key_name": "tempest-keypair-383499095", "user_data": "}} {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 27 02:26:44.554918 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.network.neutron [None req-010a9ff7-008c-4e6a-a907-489f660cef57 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] validate_networks() for [('134184f4-d632-4173-b77c-1566bf4eb0fb', None, None, None, None, None)] {{(pid=120864) validate_networks /opt/stack/nova/nova/network/neutron.py:2626}} Aug 27 02:26:44.950410 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.virt.hardware [None req-010a9ff7-008c-4e6a-a907-489f660cef57 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Flavor limits 0:0:0 {{(pid=120864) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Aug 27 02:26:44.950820 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.virt.hardware [None req-010a9ff7-008c-4e6a-a907-489f660cef57 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Image limits 0:0:0 {{(pid=120864) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Aug 27 02:26:44.951284 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.virt.hardware [None req-010a9ff7-008c-4e6a-a907-489f660cef57 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Flavor pref 0:0:0 {{(pid=120864) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Aug 27 02:26:44.951728 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.virt.hardware [None req-010a9ff7-008c-4e6a-a907-489f660cef57 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Image pref 0:0:0 {{(pid=120864) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Aug 27 02:26:44.952126 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.virt.hardware [None req-010a9ff7-008c-4e6a-a907-489f660cef57 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=120864) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Aug 27 02:26:44.963439 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-010a9ff7-008c-4e6a-a907-489f660cef57 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:26:44.963913 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-010a9ff7-008c-4e6a-a907-489f660cef57 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:26:44.964339 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-010a9ff7-008c-4e6a-a907-489f660cef57 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:26:44.965160 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-010a9ff7-008c-4e6a-a907-489f660cef57 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:26:44.965610 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-010a9ff7-008c-4e6a-a907-489f660cef57 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:26:44.966071 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-010a9ff7-008c-4e6a-a907-489f660cef57 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:26:44.986839 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.quota [None req-010a9ff7-008c-4e6a-a907-489f660cef57 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Getting quotas for project 5d5ec8450c0941d18d63558baab54737. Resources: {'cores', 'ram', 'instances'} {{(pid=120864) _get_quotas /opt/stack/nova/nova/quota.py:393}} Aug 27 02:26:44.991354 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.quota [None req-010a9ff7-008c-4e6a-a907-489f660cef57 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Getting quotas for user 26a86351583d49d090b3db09f127d49f and project 5d5ec8450c0941d18d63558baab54737. Resources: {'cores', 'ram', 'instances'} {{(pid=120864) _get_quotas /opt/stack/nova/nova/quota.py:383}} Aug 27 02:26:45.003407 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-010a9ff7-008c-4e6a-a907-489f660cef57 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Going to run 1 instances... {{(pid=120864) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Aug 27 02:26:45.005970 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-010a9ff7-008c-4e6a-a907-489f660cef57 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] [instance: dc699df0-be4f-4036-9e01-45ba0156616b] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='9fc30b36-b827-46f9-892d-30e808bc2b8a',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=120864) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Aug 27 02:26:45.057462 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-010a9ff7-008c-4e6a-a907-489f660cef57 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] 10.222.0.52 "POST /compute/v2.1/servers" status: 202 len: 401 microversion: 2.1 time: 0.888817 Aug 27 02:26:45.058732 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 263/525] 10.222.0.52 () {60 vars in 1147 bytes} [Sat Aug 27 02:26:44 2022] POST /compute/v2.1/servers => generated 401 bytes in 891 msecs (HTTP/1.1 202) 10 headers in 451 bytes (2 switches on core 0) Aug 27 02:26:45.071983 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-fdc195c4-ff6e-4a67-86a6-b52f02d1f6a2 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:26:45.087859 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-fdc195c4-ff6e-4a67-86a6-b52f02d1f6a2 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] [instance: dc699df0-be4f-4036-9e01-45ba0156616b] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:26:45.092236 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG dbcounter [-] [120863] Writing DB stats nova_cell1:SELECT=11,nova_cell1:UPDATE=2,nova_cell1:INSERT=1 {{(pid=120863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 27 02:26:45.167216 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-fdc195c4-ff6e-4a67-86a6-b52f02d1f6a2 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '26a86351583d49d090b3db09f127d49f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5d5ec8450c0941d18d63558baab54737', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:26:45.175216 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-fdc195c4-ff6e-4a67-86a6-b52f02d1f6a2 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] 10.222.0.52 "GET /compute/v2.1/servers/dc699df0-be4f-4036-9e01-45ba0156616b" status: 200 len: 1241 microversion: 2.1 time: 0.109541 Aug 27 02:26:45.176399 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 263/526] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:26:45 2022] GET /compute/v2.1/servers/dc699df0-be4f-4036-9e01-45ba0156616b => generated 1241 bytes in 111 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:26:46.199423 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-ae456a17-c0c7-4100-a8cf-6b8cccff9e3e tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Calling method '>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:26:46.200194 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-ae456a17-c0c7-4100-a8cf-6b8cccff9e3e tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] [instance: dc699df0-be4f-4036-9e01-45ba0156616b] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:26:46.207690 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-ae456a17-c0c7-4100-a8cf-6b8cccff9e3e tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:26:46.208325 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-ae456a17-c0c7-4100-a8cf-6b8cccff9e3e tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:26:46.208764 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-ae456a17-c0c7-4100-a8cf-6b8cccff9e3e tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:26:46.209542 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-ae456a17-c0c7-4100-a8cf-6b8cccff9e3e tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:26:46.210655 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-ae456a17-c0c7-4100-a8cf-6b8cccff9e3e tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:26:46.210655 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-ae456a17-c0c7-4100-a8cf-6b8cccff9e3e tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:26:46.290502 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-ae456a17-c0c7-4100-a8cf-6b8cccff9e3e tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '26a86351583d49d090b3db09f127d49f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5d5ec8450c0941d18d63558baab54737', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:26:46.299437 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-ae456a17-c0c7-4100-a8cf-6b8cccff9e3e tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] 10.222.0.52 "GET /compute/v2.1/servers/dc699df0-be4f-4036-9e01-45ba0156616b" status: 200 len: 1293 microversion: 2.1 time: 0.104304 Aug 27 02:26:46.300321 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 264/527] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:26:46 2022] GET /compute/v2.1/servers/dc699df0-be4f-4036-9e01-45ba0156616b => generated 1293 bytes in 106 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:26:47.325537 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-b9332c4a-446a-4da3-a408-f20cd66ac08c tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:26:47.327174 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-b9332c4a-446a-4da3-a408-f20cd66ac08c tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] [instance: dc699df0-be4f-4036-9e01-45ba0156616b] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:26:47.336313 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-b9332c4a-446a-4da3-a408-f20cd66ac08c tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:26:47.336313 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-b9332c4a-446a-4da3-a408-f20cd66ac08c tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:26:47.336853 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-b9332c4a-446a-4da3-a408-f20cd66ac08c tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:26:47.343045 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-b9332c4a-446a-4da3-a408-f20cd66ac08c tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:26:47.343045 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-b9332c4a-446a-4da3-a408-f20cd66ac08c tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:26:47.343045 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-b9332c4a-446a-4da3-a408-f20cd66ac08c tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:26:47.413534 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-b9332c4a-446a-4da3-a408-f20cd66ac08c tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '26a86351583d49d090b3db09f127d49f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5d5ec8450c0941d18d63558baab54737', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:26:47.421994 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-b9332c4a-446a-4da3-a408-f20cd66ac08c tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] 10.222.0.52 "GET /compute/v2.1/servers/dc699df0-be4f-4036-9e01-45ba0156616b" status: 200 len: 1299 microversion: 2.1 time: 0.104106 Aug 27 02:26:47.422573 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 264/528] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:26:47 2022] GET /compute/v2.1/servers/dc699df0-be4f-4036-9e01-45ba0156616b => generated 1299 bytes in 105 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:26:48.371784 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [req-a0e00998-c850-4156-9e2d-48753fd5d249 req-04643be7-ba0e-4ba3-95b5-be6fcb8a244c service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "dc699df0-be4f-4036-9e01-45ba0156616b", "tag": "2fe1c73f-fd8a-4318-9640-cf25059ea1c3"}]} {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 27 02:26:48.383814 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [req-a0e00998-c850-4156-9e2d-48753fd5d249 req-04643be7-ba0e-4ba3-95b5-be6fcb8a244c service nova] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:26:48.384294 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [req-a0e00998-c850-4156-9e2d-48753fd5d249 req-04643be7-ba0e-4ba3-95b5-be6fcb8a244c service nova] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:26:48.384863 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [req-a0e00998-c850-4156-9e2d-48753fd5d249 req-04643be7-ba0e-4ba3-95b5-be6fcb8a244c service nova] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:26:48.399274 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.compute.server_external_events [req-a0e00998-c850-4156-9e2d-48753fd5d249 req-04643be7-ba0e-4ba3-95b5-be6fcb8a244c service nova] Creating event network-changed:2fe1c73f-fd8a-4318-9640-cf25059ea1c3 for instance dc699df0-be4f-4036-9e01-45ba0156616b on hv-cinder-84630 Aug 27 02:26:48.404798 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [req-a0e00998-c850-4156-9e2d-48753fd5d249 req-04643be7-ba0e-4ba3-95b5-be6fcb8a244c service nova] 10.222.0.52 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.037619 Aug 27 02:26:48.405756 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 265/529] 10.222.0.52 () {64 vars in 1317 bytes} [Sat Aug 27 02:26:48 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 39 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 27 02:26:48.445278 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-a28e1298-c5c4-4fc7-9a26-49250608df08 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:26:48.446652 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-a28e1298-c5c4-4fc7-9a26-49250608df08 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] [instance: dc699df0-be4f-4036-9e01-45ba0156616b] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:26:48.455103 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-a28e1298-c5c4-4fc7-9a26-49250608df08 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:26:48.455581 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-a28e1298-c5c4-4fc7-9a26-49250608df08 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:26:48.455976 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-a28e1298-c5c4-4fc7-9a26-49250608df08 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:26:48.456918 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-a28e1298-c5c4-4fc7-9a26-49250608df08 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:26:48.457463 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-a28e1298-c5c4-4fc7-9a26-49250608df08 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:26:48.458090 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-a28e1298-c5c4-4fc7-9a26-49250608df08 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:26:48.590360 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-a28e1298-c5c4-4fc7-9a26-49250608df08 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '26a86351583d49d090b3db09f127d49f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5d5ec8450c0941d18d63558baab54737', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:26:48.598717 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-a28e1298-c5c4-4fc7-9a26-49250608df08 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] 10.222.0.52 "GET /compute/v2.1/servers/dc699df0-be4f-4036-9e01-45ba0156616b" status: 200 len: 1366 microversion: 2.1 time: 0.158432 Aug 27 02:26:48.599334 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 265/530] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:26:48 2022] GET /compute/v2.1/servers/dc699df0-be4f-4036-9e01-45ba0156616b => generated 1366 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:26:49.621629 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-04124f20-62ea-4815-9e8d-621f79bafc30 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Calling method '>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:26:49.622306 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-04124f20-62ea-4815-9e8d-621f79bafc30 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] [instance: dc699df0-be4f-4036-9e01-45ba0156616b] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:26:49.629191 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-04124f20-62ea-4815-9e8d-621f79bafc30 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:26:49.630381 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-04124f20-62ea-4815-9e8d-621f79bafc30 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:26:49.630917 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-04124f20-62ea-4815-9e8d-621f79bafc30 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:26:49.631975 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-04124f20-62ea-4815-9e8d-621f79bafc30 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:26:49.632433 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-04124f20-62ea-4815-9e8d-621f79bafc30 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:26:49.632857 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-04124f20-62ea-4815-9e8d-621f79bafc30 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:26:49.778770 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-04124f20-62ea-4815-9e8d-621f79bafc30 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '26a86351583d49d090b3db09f127d49f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5d5ec8450c0941d18d63558baab54737', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:26:49.787258 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-04124f20-62ea-4815-9e8d-621f79bafc30 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] 10.222.0.52 "GET /compute/v2.1/servers/dc699df0-be4f-4036-9e01-45ba0156616b" status: 200 len: 1366 microversion: 2.1 time: 0.170665 Aug 27 02:26:49.787825 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 266/531] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:26:49 2022] GET /compute/v2.1/servers/dc699df0-be4f-4036-9e01-45ba0156616b => generated 1366 bytes in 172 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:26:50.810090 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-195101f9-0bc1-4815-938c-fffebe3fff63 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:26:50.810731 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-195101f9-0bc1-4815-938c-fffebe3fff63 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] [instance: dc699df0-be4f-4036-9e01-45ba0156616b] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:26:50.819179 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-195101f9-0bc1-4815-938c-fffebe3fff63 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:26:50.819667 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-195101f9-0bc1-4815-938c-fffebe3fff63 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:26:50.820097 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-195101f9-0bc1-4815-938c-fffebe3fff63 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:26:50.820781 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-195101f9-0bc1-4815-938c-fffebe3fff63 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:26:50.821295 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-195101f9-0bc1-4815-938c-fffebe3fff63 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:26:50.821738 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-195101f9-0bc1-4815-938c-fffebe3fff63 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:26:50.964502 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-195101f9-0bc1-4815-938c-fffebe3fff63 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '26a86351583d49d090b3db09f127d49f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5d5ec8450c0941d18d63558baab54737', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:26:50.972769 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-195101f9-0bc1-4815-938c-fffebe3fff63 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] 10.222.0.52 "GET /compute/v2.1/servers/dc699df0-be4f-4036-9e01-45ba0156616b" status: 200 len: 1366 microversion: 2.1 time: 0.167835 Aug 27 02:26:50.975307 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 266/532] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:26:50 2022] GET /compute/v2.1/servers/dc699df0-be4f-4036-9e01-45ba0156616b => generated 1366 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:26:51.995873 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-e8539172-a884-41fa-aa86-3a41a08b0102 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Calling method '>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:26:51.996446 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-e8539172-a884-41fa-aa86-3a41a08b0102 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] [instance: dc699df0-be4f-4036-9e01-45ba0156616b] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:26:52.003630 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-e8539172-a884-41fa-aa86-3a41a08b0102 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:26:52.004164 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-e8539172-a884-41fa-aa86-3a41a08b0102 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:26:52.004589 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-e8539172-a884-41fa-aa86-3a41a08b0102 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:26:52.005399 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-e8539172-a884-41fa-aa86-3a41a08b0102 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:26:52.005840 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-e8539172-a884-41fa-aa86-3a41a08b0102 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:26:52.006255 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-e8539172-a884-41fa-aa86-3a41a08b0102 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:26:52.144331 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-e8539172-a884-41fa-aa86-3a41a08b0102 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '26a86351583d49d090b3db09f127d49f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5d5ec8450c0941d18d63558baab54737', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:26:52.160102 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-e8539172-a884-41fa-aa86-3a41a08b0102 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] 10.222.0.52 "GET /compute/v2.1/servers/dc699df0-be4f-4036-9e01-45ba0156616b" status: 200 len: 1366 microversion: 2.1 time: 0.168131 Aug 27 02:26:52.160799 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 267/533] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:26:51 2022] GET /compute/v2.1/servers/dc699df0-be4f-4036-9e01-45ba0156616b => generated 1366 bytes in 170 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:26:53.185626 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-14039ad7-c34e-4467-a909-d9178f120cba tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:26:53.186214 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-14039ad7-c34e-4467-a909-d9178f120cba tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] [instance: dc699df0-be4f-4036-9e01-45ba0156616b] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:26:53.194102 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-14039ad7-c34e-4467-a909-d9178f120cba tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:26:53.194586 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-14039ad7-c34e-4467-a909-d9178f120cba tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:26:53.195009 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-14039ad7-c34e-4467-a909-d9178f120cba tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:26:53.195692 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-14039ad7-c34e-4467-a909-d9178f120cba tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:26:53.196132 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-14039ad7-c34e-4467-a909-d9178f120cba tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:26:53.196604 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-14039ad7-c34e-4467-a909-d9178f120cba tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:26:53.337171 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-14039ad7-c34e-4467-a909-d9178f120cba tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '26a86351583d49d090b3db09f127d49f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5d5ec8450c0941d18d63558baab54737', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:26:53.345244 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-14039ad7-c34e-4467-a909-d9178f120cba tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] 10.222.0.52 "GET /compute/v2.1/servers/dc699df0-be4f-4036-9e01-45ba0156616b" status: 200 len: 1366 microversion: 2.1 time: 0.164426 Aug 27 02:26:53.345807 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 267/534] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:26:53 2022] GET /compute/v2.1/servers/dc699df0-be4f-4036-9e01-45ba0156616b => generated 1366 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:26:54.368144 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-aa667096-6cc6-4c91-b8fb-1feb6f7e521b tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Calling method '>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:26:54.368744 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-aa667096-6cc6-4c91-b8fb-1feb6f7e521b tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] [instance: dc699df0-be4f-4036-9e01-45ba0156616b] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:26:54.376381 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-aa667096-6cc6-4c91-b8fb-1feb6f7e521b tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:26:54.376869 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-aa667096-6cc6-4c91-b8fb-1feb6f7e521b tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:26:54.377342 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-aa667096-6cc6-4c91-b8fb-1feb6f7e521b tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:26:54.378036 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-aa667096-6cc6-4c91-b8fb-1feb6f7e521b tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:26:54.378480 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-aa667096-6cc6-4c91-b8fb-1feb6f7e521b tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:26:54.378892 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-aa667096-6cc6-4c91-b8fb-1feb6f7e521b tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:26:54.527259 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-aa667096-6cc6-4c91-b8fb-1feb6f7e521b tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '26a86351583d49d090b3db09f127d49f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5d5ec8450c0941d18d63558baab54737', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:26:54.536165 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-aa667096-6cc6-4c91-b8fb-1feb6f7e521b tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] 10.222.0.52 "GET /compute/v2.1/servers/dc699df0-be4f-4036-9e01-45ba0156616b" status: 200 len: 1366 microversion: 2.1 time: 0.172252 Aug 27 02:26:54.536812 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 268/535] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:26:54 2022] GET /compute/v2.1/servers/dc699df0-be4f-4036-9e01-45ba0156616b => generated 1366 bytes in 174 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:26:55.558052 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-9e2a4eed-46a3-417b-9ba6-1d5522b32410 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:26:55.558678 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-9e2a4eed-46a3-417b-9ba6-1d5522b32410 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] [instance: dc699df0-be4f-4036-9e01-45ba0156616b] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:26:55.560992 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG dbcounter [-] [120863] Writing DB stats nova_cell0:SELECT=6 {{(pid=120863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 27 02:26:55.567820 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-9e2a4eed-46a3-417b-9ba6-1d5522b32410 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:26:55.568308 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-9e2a4eed-46a3-417b-9ba6-1d5522b32410 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:26:55.568721 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-9e2a4eed-46a3-417b-9ba6-1d5522b32410 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:26:55.569439 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-9e2a4eed-46a3-417b-9ba6-1d5522b32410 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:26:55.569889 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-9e2a4eed-46a3-417b-9ba6-1d5522b32410 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:26:55.570312 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-9e2a4eed-46a3-417b-9ba6-1d5522b32410 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:26:55.718807 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-9e2a4eed-46a3-417b-9ba6-1d5522b32410 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '26a86351583d49d090b3db09f127d49f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5d5ec8450c0941d18d63558baab54737', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:26:55.728588 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-9e2a4eed-46a3-417b-9ba6-1d5522b32410 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] 10.222.0.52 "GET /compute/v2.1/servers/dc699df0-be4f-4036-9e01-45ba0156616b" status: 200 len: 1366 microversion: 2.1 time: 0.173985 Aug 27 02:26:55.729494 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 268/536] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:26:55 2022] GET /compute/v2.1/servers/dc699df0-be4f-4036-9e01-45ba0156616b => generated 1366 bytes in 176 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:26:56.753163 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-7eef8d33-a8a5-4fc9-8a48-9ff72417d075 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Calling method '>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:26:56.753824 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-7eef8d33-a8a5-4fc9-8a48-9ff72417d075 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] [instance: dc699df0-be4f-4036-9e01-45ba0156616b] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:26:56.756217 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG dbcounter [-] [120864] Writing DB stats nova_cell0:SELECT=2 {{(pid=120864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 27 02:26:56.756960 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG dbcounter [-] [120864] Writing DB stats nova_cell0:SELECT=6 {{(pid=120864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 27 02:26:56.764222 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-7eef8d33-a8a5-4fc9-8a48-9ff72417d075 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:26:56.764956 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-7eef8d33-a8a5-4fc9-8a48-9ff72417d075 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:26:56.765591 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-7eef8d33-a8a5-4fc9-8a48-9ff72417d075 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:26:56.766290 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-7eef8d33-a8a5-4fc9-8a48-9ff72417d075 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:26:56.766757 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-7eef8d33-a8a5-4fc9-8a48-9ff72417d075 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:26:56.767215 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-7eef8d33-a8a5-4fc9-8a48-9ff72417d075 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:26:56.926173 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-7eef8d33-a8a5-4fc9-8a48-9ff72417d075 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '26a86351583d49d090b3db09f127d49f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5d5ec8450c0941d18d63558baab54737', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:26:56.934618 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-7eef8d33-a8a5-4fc9-8a48-9ff72417d075 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] 10.222.0.52 "GET /compute/v2.1/servers/dc699df0-be4f-4036-9e01-45ba0156616b" status: 200 len: 1366 microversion: 2.1 time: 0.185694 Aug 27 02:26:56.935217 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 269/537] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:26:56 2022] GET /compute/v2.1/servers/dc699df0-be4f-4036-9e01-45ba0156616b => generated 1366 bytes in 187 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:26:57.958353 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-041f4070-2103-4c92-b8e4-eb7a978cfeed tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:26:57.958984 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-041f4070-2103-4c92-b8e4-eb7a978cfeed tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] [instance: dc699df0-be4f-4036-9e01-45ba0156616b] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:26:57.965705 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-041f4070-2103-4c92-b8e4-eb7a978cfeed tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:26:57.966190 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-041f4070-2103-4c92-b8e4-eb7a978cfeed tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:26:57.966681 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-041f4070-2103-4c92-b8e4-eb7a978cfeed tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:26:57.967327 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-041f4070-2103-4c92-b8e4-eb7a978cfeed tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:26:57.967806 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-041f4070-2103-4c92-b8e4-eb7a978cfeed tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:26:57.968228 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-041f4070-2103-4c92-b8e4-eb7a978cfeed tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:26:58.111180 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-041f4070-2103-4c92-b8e4-eb7a978cfeed tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '26a86351583d49d090b3db09f127d49f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5d5ec8450c0941d18d63558baab54737', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:26:58.120098 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-041f4070-2103-4c92-b8e4-eb7a978cfeed tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] 10.222.0.52 "GET /compute/v2.1/servers/dc699df0-be4f-4036-9e01-45ba0156616b" status: 200 len: 1366 microversion: 2.1 time: 0.165576 Aug 27 02:26:58.120676 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 269/538] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:26:57 2022] GET /compute/v2.1/servers/dc699df0-be4f-4036-9e01-45ba0156616b => generated 1366 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:26:59.143388 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-24273e48-30e9-4c0f-9642-bef51abecb1b tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Calling method '>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:26:59.144040 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-24273e48-30e9-4c0f-9642-bef51abecb1b tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] [instance: dc699df0-be4f-4036-9e01-45ba0156616b] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:26:59.153853 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-24273e48-30e9-4c0f-9642-bef51abecb1b tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:26:59.154586 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-24273e48-30e9-4c0f-9642-bef51abecb1b tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:26:59.155153 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-24273e48-30e9-4c0f-9642-bef51abecb1b tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:26:59.156059 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-24273e48-30e9-4c0f-9642-bef51abecb1b tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:26:59.156651 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-24273e48-30e9-4c0f-9642-bef51abecb1b tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:26:59.157301 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-24273e48-30e9-4c0f-9642-bef51abecb1b tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:26:59.311616 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-24273e48-30e9-4c0f-9642-bef51abecb1b tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '26a86351583d49d090b3db09f127d49f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5d5ec8450c0941d18d63558baab54737', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:26:59.319870 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-24273e48-30e9-4c0f-9642-bef51abecb1b tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] 10.222.0.52 "GET /compute/v2.1/servers/dc699df0-be4f-4036-9e01-45ba0156616b" status: 200 len: 1366 microversion: 2.1 time: 0.181233 Aug 27 02:26:59.320458 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 270/539] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:26:59 2022] GET /compute/v2.1/servers/dc699df0-be4f-4036-9e01-45ba0156616b => generated 1366 bytes in 183 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:27:00.340312 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-c605f93b-57aa-4df7-b972-98cc3fa6d079 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:27:00.340905 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-c605f93b-57aa-4df7-b972-98cc3fa6d079 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] [instance: dc699df0-be4f-4036-9e01-45ba0156616b] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:27:00.347732 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-c605f93b-57aa-4df7-b972-98cc3fa6d079 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:27:00.348359 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-c605f93b-57aa-4df7-b972-98cc3fa6d079 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:27:00.348912 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-c605f93b-57aa-4df7-b972-98cc3fa6d079 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:27:00.349654 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-c605f93b-57aa-4df7-b972-98cc3fa6d079 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:27:00.350110 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-c605f93b-57aa-4df7-b972-98cc3fa6d079 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:27:00.350867 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-c605f93b-57aa-4df7-b972-98cc3fa6d079 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:27:00.483815 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-c605f93b-57aa-4df7-b972-98cc3fa6d079 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '26a86351583d49d090b3db09f127d49f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5d5ec8450c0941d18d63558baab54737', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:27:00.492456 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-c605f93b-57aa-4df7-b972-98cc3fa6d079 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] 10.222.0.52 "GET /compute/v2.1/servers/dc699df0-be4f-4036-9e01-45ba0156616b" status: 200 len: 1366 microversion: 2.1 time: 0.155949 Aug 27 02:27:00.493033 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 270/540] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:27:00 2022] GET /compute/v2.1/servers/dc699df0-be4f-4036-9e01-45ba0156616b => generated 1366 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:27:01.514627 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-f09d478b-793a-446a-8b39-73b85068a8f5 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Calling method '>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:27:01.515258 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-f09d478b-793a-446a-8b39-73b85068a8f5 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] [instance: dc699df0-be4f-4036-9e01-45ba0156616b] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:27:01.517667 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG dbcounter [-] [120864] Writing DB stats nova_api:SELECT=34,nova_api:INSERT=3 {{(pid=120864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 27 02:27:01.524052 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-f09d478b-793a-446a-8b39-73b85068a8f5 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:27:01.524537 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-f09d478b-793a-446a-8b39-73b85068a8f5 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:27:01.525033 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-f09d478b-793a-446a-8b39-73b85068a8f5 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:27:01.525899 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-f09d478b-793a-446a-8b39-73b85068a8f5 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:27:01.526359 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-f09d478b-793a-446a-8b39-73b85068a8f5 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:27:01.526782 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-f09d478b-793a-446a-8b39-73b85068a8f5 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:27:01.531190 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG dbcounter [-] [120864] Writing DB stats nova_cell1:SELECT=56,nova_cell1:UPDATE=2,nova_cell1:INSERT=1 {{(pid=120864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 27 02:27:01.671586 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-f09d478b-793a-446a-8b39-73b85068a8f5 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '26a86351583d49d090b3db09f127d49f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5d5ec8450c0941d18d63558baab54737', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:27:01.681643 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-f09d478b-793a-446a-8b39-73b85068a8f5 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] 10.222.0.52 "GET /compute/v2.1/servers/dc699df0-be4f-4036-9e01-45ba0156616b" status: 200 len: 1366 microversion: 2.1 time: 0.171355 Aug 27 02:27:01.682230 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 271/541] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:27:01 2022] GET /compute/v2.1/servers/dc699df0-be4f-4036-9e01-45ba0156616b => generated 1366 bytes in 173 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:27:02.710398 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-34e01d96-c005-42b0-9fd3-27ccf55a3de7 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:27:02.711017 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-34e01d96-c005-42b0-9fd3-27ccf55a3de7 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] [instance: dc699df0-be4f-4036-9e01-45ba0156616b] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:27:02.717935 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-34e01d96-c005-42b0-9fd3-27ccf55a3de7 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:27:02.718429 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-34e01d96-c005-42b0-9fd3-27ccf55a3de7 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:27:02.718857 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-34e01d96-c005-42b0-9fd3-27ccf55a3de7 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:27:02.719558 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-34e01d96-c005-42b0-9fd3-27ccf55a3de7 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:27:02.720011 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-34e01d96-c005-42b0-9fd3-27ccf55a3de7 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:27:02.720435 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-34e01d96-c005-42b0-9fd3-27ccf55a3de7 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:27:02.863597 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-34e01d96-c005-42b0-9fd3-27ccf55a3de7 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '26a86351583d49d090b3db09f127d49f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5d5ec8450c0941d18d63558baab54737', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:27:02.872061 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-34e01d96-c005-42b0-9fd3-27ccf55a3de7 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] 10.222.0.52 "GET /compute/v2.1/servers/dc699df0-be4f-4036-9e01-45ba0156616b" status: 200 len: 1366 microversion: 2.1 time: 0.166250 Aug 27 02:27:02.872631 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 271/542] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:27:02 2022] GET /compute/v2.1/servers/dc699df0-be4f-4036-9e01-45ba0156616b => generated 1366 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:27:03.895450 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-33ab15c7-dd72-45c9-a0c6-5adb84e31403 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Calling method '>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:27:03.896140 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-33ab15c7-dd72-45c9-a0c6-5adb84e31403 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] [instance: dc699df0-be4f-4036-9e01-45ba0156616b] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:27:03.902983 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-33ab15c7-dd72-45c9-a0c6-5adb84e31403 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:27:03.903462 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-33ab15c7-dd72-45c9-a0c6-5adb84e31403 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:27:03.903898 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-33ab15c7-dd72-45c9-a0c6-5adb84e31403 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:27:03.904585 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-33ab15c7-dd72-45c9-a0c6-5adb84e31403 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:27:03.905034 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-33ab15c7-dd72-45c9-a0c6-5adb84e31403 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:27:03.906931 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-33ab15c7-dd72-45c9-a0c6-5adb84e31403 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:27:04.048238 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-33ab15c7-dd72-45c9-a0c6-5adb84e31403 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '26a86351583d49d090b3db09f127d49f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5d5ec8450c0941d18d63558baab54737', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:27:04.057237 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-33ab15c7-dd72-45c9-a0c6-5adb84e31403 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] 10.222.0.52 "GET /compute/v2.1/servers/dc699df0-be4f-4036-9e01-45ba0156616b" status: 200 len: 1366 microversion: 2.1 time: 0.165770 Aug 27 02:27:04.057810 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 272/543] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:27:03 2022] GET /compute/v2.1/servers/dc699df0-be4f-4036-9e01-45ba0156616b => generated 1366 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:27:05.081072 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-f835ef61-c7a7-40c5-8ec8-a51b6c6adfc7 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:27:05.081726 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-f835ef61-c7a7-40c5-8ec8-a51b6c6adfc7 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] [instance: dc699df0-be4f-4036-9e01-45ba0156616b] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:27:05.084544 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG dbcounter [-] [120863] Writing DB stats nova_api:SELECT=34,nova_api:INSERT=1 {{(pid=120863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 27 02:27:05.090229 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-f835ef61-c7a7-40c5-8ec8-a51b6c6adfc7 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:27:05.090712 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-f835ef61-c7a7-40c5-8ec8-a51b6c6adfc7 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:27:05.091144 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-f835ef61-c7a7-40c5-8ec8-a51b6c6adfc7 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:27:05.091840 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-f835ef61-c7a7-40c5-8ec8-a51b6c6adfc7 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:27:05.092295 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-f835ef61-c7a7-40c5-8ec8-a51b6c6adfc7 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:27:05.092761 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-f835ef61-c7a7-40c5-8ec8-a51b6c6adfc7 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:27:05.233528 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-f835ef61-c7a7-40c5-8ec8-a51b6c6adfc7 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '26a86351583d49d090b3db09f127d49f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5d5ec8450c0941d18d63558baab54737', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:27:05.241765 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-f835ef61-c7a7-40c5-8ec8-a51b6c6adfc7 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] 10.222.0.52 "GET /compute/v2.1/servers/dc699df0-be4f-4036-9e01-45ba0156616b" status: 200 len: 1366 microversion: 2.1 time: 0.164908 Aug 27 02:27:05.242417 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 272/544] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:27:05 2022] GET /compute/v2.1/servers/dc699df0-be4f-4036-9e01-45ba0156616b => generated 1366 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:27:06.269020 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-d625de29-a3c3-4e3c-988a-3a7dab6d9b4b tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Calling method '>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:27:06.269678 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-d625de29-a3c3-4e3c-988a-3a7dab6d9b4b tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] [instance: dc699df0-be4f-4036-9e01-45ba0156616b] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:27:06.276695 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-d625de29-a3c3-4e3c-988a-3a7dab6d9b4b tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:27:06.277231 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-d625de29-a3c3-4e3c-988a-3a7dab6d9b4b tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:27:06.277664 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-d625de29-a3c3-4e3c-988a-3a7dab6d9b4b tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:27:06.278364 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-d625de29-a3c3-4e3c-988a-3a7dab6d9b4b tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:27:06.278785 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-d625de29-a3c3-4e3c-988a-3a7dab6d9b4b tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:27:06.279211 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-d625de29-a3c3-4e3c-988a-3a7dab6d9b4b tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:27:06.425824 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-d625de29-a3c3-4e3c-988a-3a7dab6d9b4b tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '26a86351583d49d090b3db09f127d49f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5d5ec8450c0941d18d63558baab54737', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:27:06.434762 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-d625de29-a3c3-4e3c-988a-3a7dab6d9b4b tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] 10.222.0.52 "GET /compute/v2.1/servers/dc699df0-be4f-4036-9e01-45ba0156616b" status: 200 len: 1366 microversion: 2.1 time: 0.169661 Aug 27 02:27:06.435303 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 273/545] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:27:06 2022] GET /compute/v2.1/servers/dc699df0-be4f-4036-9e01-45ba0156616b => generated 1366 bytes in 176 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:27:07.456289 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-56a46c32-b5e9-48b3-b935-fdcd4387acb3 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:27:07.456983 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-56a46c32-b5e9-48b3-b935-fdcd4387acb3 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] [instance: dc699df0-be4f-4036-9e01-45ba0156616b] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:27:07.464165 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-56a46c32-b5e9-48b3-b935-fdcd4387acb3 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:27:07.464658 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-56a46c32-b5e9-48b3-b935-fdcd4387acb3 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:27:07.465092 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-56a46c32-b5e9-48b3-b935-fdcd4387acb3 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:27:07.465826 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-56a46c32-b5e9-48b3-b935-fdcd4387acb3 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:27:07.466287 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-56a46c32-b5e9-48b3-b935-fdcd4387acb3 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:27:07.466715 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-56a46c32-b5e9-48b3-b935-fdcd4387acb3 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:27:07.625814 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-56a46c32-b5e9-48b3-b935-fdcd4387acb3 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '26a86351583d49d090b3db09f127d49f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5d5ec8450c0941d18d63558baab54737', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:27:07.634245 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-56a46c32-b5e9-48b3-b935-fdcd4387acb3 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] 10.222.0.52 "GET /compute/v2.1/servers/dc699df0-be4f-4036-9e01-45ba0156616b" status: 200 len: 1366 microversion: 2.1 time: 0.182131 Aug 27 02:27:07.634797 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 273/546] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:27:07 2022] GET /compute/v2.1/servers/dc699df0-be4f-4036-9e01-45ba0156616b => generated 1366 bytes in 183 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:27:08.657603 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-6a8e2520-78e7-436d-aaae-77df74bb7744 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Calling method '>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:27:08.657603 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-6a8e2520-78e7-436d-aaae-77df74bb7744 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] [instance: dc699df0-be4f-4036-9e01-45ba0156616b] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:27:08.663059 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-6a8e2520-78e7-436d-aaae-77df74bb7744 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:27:08.663544 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-6a8e2520-78e7-436d-aaae-77df74bb7744 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:27:08.663975 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-6a8e2520-78e7-436d-aaae-77df74bb7744 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:27:08.664657 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-6a8e2520-78e7-436d-aaae-77df74bb7744 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:27:08.665102 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-6a8e2520-78e7-436d-aaae-77df74bb7744 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:27:08.665559 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-6a8e2520-78e7-436d-aaae-77df74bb7744 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:27:08.816327 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-6a8e2520-78e7-436d-aaae-77df74bb7744 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '26a86351583d49d090b3db09f127d49f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5d5ec8450c0941d18d63558baab54737', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:27:08.824769 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-6a8e2520-78e7-436d-aaae-77df74bb7744 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] 10.222.0.52 "GET /compute/v2.1/servers/dc699df0-be4f-4036-9e01-45ba0156616b" status: 200 len: 1366 microversion: 2.1 time: 0.172667 Aug 27 02:27:08.825380 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 274/547] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:27:08 2022] GET /compute/v2.1/servers/dc699df0-be4f-4036-9e01-45ba0156616b => generated 1366 bytes in 174 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:27:09.846505 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-1a5bce10-9e93-4495-8da6-70263a7b04a3 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:27:09.847296 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-1a5bce10-9e93-4495-8da6-70263a7b04a3 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] [instance: dc699df0-be4f-4036-9e01-45ba0156616b] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:27:09.854721 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-1a5bce10-9e93-4495-8da6-70263a7b04a3 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:27:09.855208 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-1a5bce10-9e93-4495-8da6-70263a7b04a3 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:27:09.855641 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-1a5bce10-9e93-4495-8da6-70263a7b04a3 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:27:09.856332 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-1a5bce10-9e93-4495-8da6-70263a7b04a3 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:27:09.856779 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-1a5bce10-9e93-4495-8da6-70263a7b04a3 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:27:09.857236 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-1a5bce10-9e93-4495-8da6-70263a7b04a3 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:27:09.993137 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-1a5bce10-9e93-4495-8da6-70263a7b04a3 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '26a86351583d49d090b3db09f127d49f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5d5ec8450c0941d18d63558baab54737', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:27:10.001204 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-1a5bce10-9e93-4495-8da6-70263a7b04a3 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] 10.222.0.52 "GET /compute/v2.1/servers/dc699df0-be4f-4036-9e01-45ba0156616b" status: 200 len: 1366 microversion: 2.1 time: 0.158790 Aug 27 02:27:10.001794 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 274/548] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:27:09 2022] GET /compute/v2.1/servers/dc699df0-be4f-4036-9e01-45ba0156616b => generated 1366 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:27:11.030460 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-dd1ddd4c-33cc-4f0a-b0f6-7907167825ed tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Calling method '>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:27:11.031066 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-dd1ddd4c-33cc-4f0a-b0f6-7907167825ed tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] [instance: dc699df0-be4f-4036-9e01-45ba0156616b] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:27:11.039858 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-dd1ddd4c-33cc-4f0a-b0f6-7907167825ed tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:27:11.040346 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-dd1ddd4c-33cc-4f0a-b0f6-7907167825ed tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:27:11.040773 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-dd1ddd4c-33cc-4f0a-b0f6-7907167825ed tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:27:11.041508 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-dd1ddd4c-33cc-4f0a-b0f6-7907167825ed tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:27:11.042018 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-dd1ddd4c-33cc-4f0a-b0f6-7907167825ed tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:27:11.042408 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-dd1ddd4c-33cc-4f0a-b0f6-7907167825ed tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:27:11.189091 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-dd1ddd4c-33cc-4f0a-b0f6-7907167825ed tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '26a86351583d49d090b3db09f127d49f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5d5ec8450c0941d18d63558baab54737', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:27:11.197046 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-dd1ddd4c-33cc-4f0a-b0f6-7907167825ed tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] 10.222.0.52 "GET /compute/v2.1/servers/dc699df0-be4f-4036-9e01-45ba0156616b" status: 200 len: 1366 microversion: 2.1 time: 0.172789 Aug 27 02:27:11.197642 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 275/549] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:27:11 2022] GET /compute/v2.1/servers/dc699df0-be4f-4036-9e01-45ba0156616b => generated 1366 bytes in 174 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:27:12.222651 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-67cb61e4-8f52-48e9-af7e-7c7c6f19f36f tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:27:12.223738 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-67cb61e4-8f52-48e9-af7e-7c7c6f19f36f tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] [instance: dc699df0-be4f-4036-9e01-45ba0156616b] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:27:12.234651 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-67cb61e4-8f52-48e9-af7e-7c7c6f19f36f tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:27:12.235448 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-67cb61e4-8f52-48e9-af7e-7c7c6f19f36f tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:27:12.236169 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-67cb61e4-8f52-48e9-af7e-7c7c6f19f36f tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:27:12.237394 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-67cb61e4-8f52-48e9-af7e-7c7c6f19f36f tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:27:12.238134 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-67cb61e4-8f52-48e9-af7e-7c7c6f19f36f tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:27:12.238928 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-67cb61e4-8f52-48e9-af7e-7c7c6f19f36f tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:27:12.392967 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-67cb61e4-8f52-48e9-af7e-7c7c6f19f36f tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '26a86351583d49d090b3db09f127d49f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5d5ec8450c0941d18d63558baab54737', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:27:12.401331 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-67cb61e4-8f52-48e9-af7e-7c7c6f19f36f tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] 10.222.0.52 "GET /compute/v2.1/servers/dc699df0-be4f-4036-9e01-45ba0156616b" status: 200 len: 1366 microversion: 2.1 time: 0.185996 Aug 27 02:27:12.401914 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 275/550] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:27:12 2022] GET /compute/v2.1/servers/dc699df0-be4f-4036-9e01-45ba0156616b => generated 1366 bytes in 187 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:27:13.422612 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-f579ec71-a871-4f56-9cad-df63b4a454fb tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Calling method '>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:27:13.423319 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-f579ec71-a871-4f56-9cad-df63b4a454fb tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] [instance: dc699df0-be4f-4036-9e01-45ba0156616b] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:27:13.430997 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-f579ec71-a871-4f56-9cad-df63b4a454fb tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:27:13.431475 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-f579ec71-a871-4f56-9cad-df63b4a454fb tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:27:13.431900 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-f579ec71-a871-4f56-9cad-df63b4a454fb tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:27:13.432585 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-f579ec71-a871-4f56-9cad-df63b4a454fb tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:27:13.433026 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-f579ec71-a871-4f56-9cad-df63b4a454fb tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:27:13.433462 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-f579ec71-a871-4f56-9cad-df63b4a454fb tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:27:13.577452 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-f579ec71-a871-4f56-9cad-df63b4a454fb tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '26a86351583d49d090b3db09f127d49f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5d5ec8450c0941d18d63558baab54737', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:27:13.589936 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-f579ec71-a871-4f56-9cad-df63b4a454fb tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] 10.222.0.52 "GET /compute/v2.1/servers/dc699df0-be4f-4036-9e01-45ba0156616b" status: 200 len: 1366 microversion: 2.1 time: 0.171581 Aug 27 02:27:13.590774 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 276/551] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:27:13 2022] GET /compute/v2.1/servers/dc699df0-be4f-4036-9e01-45ba0156616b => generated 1366 bytes in 173 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:27:13.801931 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [req-cf4176b3-a7c6-4aba-95b8-bf7e69d0b9a9 req-daa1e2b7-6111-440b-b112-78825e6de6bb service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "dc699df0-be4f-4036-9e01-45ba0156616b", "name": "network-vif-plugged", "status": "completed", "tag": "2fe1c73f-fd8a-4318-9640-cf25059ea1c3"}]} {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 27 02:27:13.812422 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [req-cf4176b3-a7c6-4aba-95b8-bf7e69d0b9a9 req-daa1e2b7-6111-440b-b112-78825e6de6bb service nova] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:27:13.812882 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [req-cf4176b3-a7c6-4aba-95b8-bf7e69d0b9a9 req-daa1e2b7-6111-440b-b112-78825e6de6bb service nova] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:27:13.813341 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [req-cf4176b3-a7c6-4aba-95b8-bf7e69d0b9a9 req-daa1e2b7-6111-440b-b112-78825e6de6bb service nova] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:27:13.826600 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.compute.server_external_events [req-cf4176b3-a7c6-4aba-95b8-bf7e69d0b9a9 req-daa1e2b7-6111-440b-b112-78825e6de6bb service nova] Creating event network-vif-plugged:2fe1c73f-fd8a-4318-9640-cf25059ea1c3 for instance dc699df0-be4f-4036-9e01-45ba0156616b on hv-cinder-84630 Aug 27 02:27:13.833744 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [req-cf4176b3-a7c6-4aba-95b8-bf7e69d0b9a9 req-daa1e2b7-6111-440b-b112-78825e6de6bb service nova] 10.222.0.52 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.034748 Aug 27 02:27:13.834521 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 276/552] 10.222.0.52 () {64 vars in 1317 bytes} [Sat Aug 27 02:27:13 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 37 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 27 02:27:14.615388 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-93f150ed-5db3-41be-ada4-82bf1bc852ca tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Calling method '>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:27:14.616130 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-93f150ed-5db3-41be-ada4-82bf1bc852ca tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] [instance: dc699df0-be4f-4036-9e01-45ba0156616b] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:27:14.622884 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-93f150ed-5db3-41be-ada4-82bf1bc852ca tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:27:14.623362 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-93f150ed-5db3-41be-ada4-82bf1bc852ca tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:27:14.623856 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-93f150ed-5db3-41be-ada4-82bf1bc852ca tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:27:14.624553 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-93f150ed-5db3-41be-ada4-82bf1bc852ca tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:27:14.625020 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-93f150ed-5db3-41be-ada4-82bf1bc852ca tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:27:14.625477 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-93f150ed-5db3-41be-ada4-82bf1bc852ca tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:27:14.787861 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-93f150ed-5db3-41be-ada4-82bf1bc852ca tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '26a86351583d49d090b3db09f127d49f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5d5ec8450c0941d18d63558baab54737', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:27:14.796085 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-93f150ed-5db3-41be-ada4-82bf1bc852ca tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] 10.222.0.52 "GET /compute/v2.1/servers/dc699df0-be4f-4036-9e01-45ba0156616b" status: 200 len: 1366 microversion: 2.1 time: 0.185973 Aug 27 02:27:14.796714 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 277/553] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:27:14 2022] GET /compute/v2.1/servers/dc699df0-be4f-4036-9e01-45ba0156616b => generated 1366 bytes in 188 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:27:15.819638 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-9034aab5-daad-474d-9197-5ba42c4bb26f tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:27:15.820359 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-9034aab5-daad-474d-9197-5ba42c4bb26f tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] [instance: dc699df0-be4f-4036-9e01-45ba0156616b] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:27:15.827237 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-9034aab5-daad-474d-9197-5ba42c4bb26f tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:27:15.827725 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-9034aab5-daad-474d-9197-5ba42c4bb26f tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:27:15.828147 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-9034aab5-daad-474d-9197-5ba42c4bb26f tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:27:15.829528 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-9034aab5-daad-474d-9197-5ba42c4bb26f tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:27:15.829687 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-9034aab5-daad-474d-9197-5ba42c4bb26f tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:27:15.830079 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-9034aab5-daad-474d-9197-5ba42c4bb26f tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:27:15.835023 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG dbcounter [-] [120863] Writing DB stats nova_cell1:SELECT=75 {{(pid=120863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 27 02:27:15.976258 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-9034aab5-daad-474d-9197-5ba42c4bb26f tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '26a86351583d49d090b3db09f127d49f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5d5ec8450c0941d18d63558baab54737', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:27:15.987288 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-9034aab5-daad-474d-9197-5ba42c4bb26f tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] 10.222.0.52 "GET /compute/v2.1/servers/dc699df0-be4f-4036-9e01-45ba0156616b" status: 200 len: 1366 microversion: 2.1 time: 0.172652 Aug 27 02:27:15.990799 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 277/554] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:27:15 2022] GET /compute/v2.1/servers/dc699df0-be4f-4036-9e01-45ba0156616b => generated 1366 bytes in 177 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:27:17.008337 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-a6310be7-c474-472d-a011-b57b90b80fa5 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Calling method '>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:27:17.009028 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-a6310be7-c474-472d-a011-b57b90b80fa5 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] [instance: dc699df0-be4f-4036-9e01-45ba0156616b] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:27:17.016135 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-a6310be7-c474-472d-a011-b57b90b80fa5 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:27:17.016748 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-a6310be7-c474-472d-a011-b57b90b80fa5 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:27:17.017365 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-a6310be7-c474-472d-a011-b57b90b80fa5 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:27:17.018481 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-a6310be7-c474-472d-a011-b57b90b80fa5 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:27:17.019060 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-a6310be7-c474-472d-a011-b57b90b80fa5 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:27:17.019501 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-a6310be7-c474-472d-a011-b57b90b80fa5 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:27:17.158135 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-a6310be7-c474-472d-a011-b57b90b80fa5 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '26a86351583d49d090b3db09f127d49f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5d5ec8450c0941d18d63558baab54737', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:27:17.166268 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-a6310be7-c474-472d-a011-b57b90b80fa5 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] 10.222.0.52 "GET /compute/v2.1/servers/dc699df0-be4f-4036-9e01-45ba0156616b" status: 200 len: 1555 microversion: 2.1 time: 0.162041 Aug 27 02:27:17.166827 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 278/555] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:27:17 2022] GET /compute/v2.1/servers/dc699df0-be4f-4036-9e01-45ba0156616b => generated 1555 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:27:17.186961 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-ccaf38c6-6b31-44be-affc-09ba07d4dd3a tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:27:17.187596 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-ccaf38c6-6b31-44be-affc-09ba07d4dd3a tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] [instance: dc699df0-be4f-4036-9e01-45ba0156616b] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:27:17.194420 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-ccaf38c6-6b31-44be-affc-09ba07d4dd3a tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:27:17.194902 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-ccaf38c6-6b31-44be-affc-09ba07d4dd3a tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:27:17.195332 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-ccaf38c6-6b31-44be-affc-09ba07d4dd3a tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:27:17.196020 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-ccaf38c6-6b31-44be-affc-09ba07d4dd3a tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:27:17.196468 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-ccaf38c6-6b31-44be-affc-09ba07d4dd3a tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:27:17.196884 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-ccaf38c6-6b31-44be-affc-09ba07d4dd3a tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:27:17.279661 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-ccaf38c6-6b31-44be-affc-09ba07d4dd3a tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] 10.222.0.52 "GET /compute/v2.1/servers/dc699df0-be4f-4036-9e01-45ba0156616b/os-interface" status: 200 len: 284 microversion: 2.1 time: 0.096687 Aug 27 02:27:17.280255 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 278/556] 10.222.0.52 () {58 vars in 1275 bytes} [Sat Aug 27 02:27:17 2022] GET /compute/v2.1/servers/dc699df0-be4f-4036-9e01-45ba0156616b/os-interface => generated 284 bytes in 98 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 27 02:27:17.337564 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-8d43e543-599e-4414-9794-4150225f8139 tempest-VolumesSnapshotTestJSON-1750456804 tempest-VolumesSnapshotTestJSON-1750456804-project] Calling method '>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:27:17.338059 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-8d43e543-599e-4414-9794-4150225f8139 tempest-VolumesSnapshotTestJSON-1750456804 tempest-VolumesSnapshotTestJSON-1750456804-project] [instance: c31c26a8-fe5a-40fb-91a1-7b5251fd0daa] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:27:17.344244 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-8d43e543-599e-4414-9794-4150225f8139 tempest-VolumesSnapshotTestJSON-1750456804 tempest-VolumesSnapshotTestJSON-1750456804-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:27:17.344730 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-8d43e543-599e-4414-9794-4150225f8139 tempest-VolumesSnapshotTestJSON-1750456804 tempest-VolumesSnapshotTestJSON-1750456804-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:27:17.345182 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-8d43e543-599e-4414-9794-4150225f8139 tempest-VolumesSnapshotTestJSON-1750456804 tempest-VolumesSnapshotTestJSON-1750456804-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:27:17.345888 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-8d43e543-599e-4414-9794-4150225f8139 tempest-VolumesSnapshotTestJSON-1750456804 tempest-VolumesSnapshotTestJSON-1750456804-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:27:17.346340 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-8d43e543-599e-4414-9794-4150225f8139 tempest-VolumesSnapshotTestJSON-1750456804 tempest-VolumesSnapshotTestJSON-1750456804-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:27:17.346759 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-8d43e543-599e-4414-9794-4150225f8139 tempest-VolumesSnapshotTestJSON-1750456804 tempest-VolumesSnapshotTestJSON-1750456804-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:27:17.484027 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.objects.instance [None req-8d43e543-599e-4414-9794-4150225f8139 tempest-VolumesSnapshotTestJSON-1750456804 tempest-VolumesSnapshotTestJSON-1750456804-project] Lazy-loading 'services' on Instance uuid c31c26a8-fe5a-40fb-91a1-7b5251fd0daa {{(pid=120864) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Aug 27 02:27:17.607264 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-8d43e543-599e-4414-9794-4150225f8139 tempest-VolumesSnapshotTestJSON-1750456804 tempest-VolumesSnapshotTestJSON-1750456804-project] 10.222.0.52 "DELETE /compute/v2.1/servers/c31c26a8-fe5a-40fb-91a1-7b5251fd0daa/os-volume_attachments/e3c6fdb0-e4d8-4f19-b6f8-e585b9ba482c" status: 202 len: 0 microversion: 2.1 time: 0.273480 Aug 27 02:27:17.607852 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 279/557] 10.222.0.52 () {58 vars in 1416 bytes} [Sat Aug 27 02:27:17 2022] DELETE /compute/v2.1/servers/c31c26a8-fe5a-40fb-91a1-7b5251fd0daa/os-volume_attachments/e3c6fdb0-e4d8-4f19-b6f8-e585b9ba482c => generated 0 bytes in 275 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Aug 27 02:27:18.286418 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [req-9141d500-e818-410b-9e8d-4ce9b7d35064 req-bd1c4167-f677-4d14-a3d7-bf3a16a859af service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "dc699df0-be4f-4036-9e01-45ba0156616b", "tag": "2fe1c73f-fd8a-4318-9640-cf25059ea1c3"}]} {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 27 02:27:18.298962 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [req-9141d500-e818-410b-9e8d-4ce9b7d35064 req-bd1c4167-f677-4d14-a3d7-bf3a16a859af service nova] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:27:18.299436 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [req-9141d500-e818-410b-9e8d-4ce9b7d35064 req-bd1c4167-f677-4d14-a3d7-bf3a16a859af service nova] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:27:18.299888 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [req-9141d500-e818-410b-9e8d-4ce9b7d35064 req-bd1c4167-f677-4d14-a3d7-bf3a16a859af service nova] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:27:18.312104 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.compute.server_external_events [req-9141d500-e818-410b-9e8d-4ce9b7d35064 req-bd1c4167-f677-4d14-a3d7-bf3a16a859af service nova] Creating event network-changed:2fe1c73f-fd8a-4318-9640-cf25059ea1c3 for instance dc699df0-be4f-4036-9e01-45ba0156616b on hv-cinder-84630 Aug 27 02:27:18.319954 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [req-9141d500-e818-410b-9e8d-4ce9b7d35064 req-bd1c4167-f677-4d14-a3d7-bf3a16a859af service nova] 10.222.0.52 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.037329 Aug 27 02:27:18.321433 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 279/558] 10.222.0.52 () {64 vars in 1317 bytes} [Sat Aug 27 02:27:18 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 40 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 27 02:27:19.836362 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-4a754171-5f3d-44ef-99c7-dad8b761e470 tempest-VolumesSnapshotTestJSON-1750456804 tempest-VolumesSnapshotTestJSON-1750456804-project] Calling method '>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:27:19.836934 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-4a754171-5f3d-44ef-99c7-dad8b761e470 tempest-VolumesSnapshotTestJSON-1750456804 tempest-VolumesSnapshotTestJSON-1750456804-project] [instance: c31c26a8-fe5a-40fb-91a1-7b5251fd0daa] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:27:19.843277 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-4a754171-5f3d-44ef-99c7-dad8b761e470 tempest-VolumesSnapshotTestJSON-1750456804 tempest-VolumesSnapshotTestJSON-1750456804-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:27:19.843760 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-4a754171-5f3d-44ef-99c7-dad8b761e470 tempest-VolumesSnapshotTestJSON-1750456804 tempest-VolumesSnapshotTestJSON-1750456804-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:27:19.844191 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-4a754171-5f3d-44ef-99c7-dad8b761e470 tempest-VolumesSnapshotTestJSON-1750456804 tempest-VolumesSnapshotTestJSON-1750456804-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:27:19.844879 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-4a754171-5f3d-44ef-99c7-dad8b761e470 tempest-VolumesSnapshotTestJSON-1750456804 tempest-VolumesSnapshotTestJSON-1750456804-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:27:19.845395 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-4a754171-5f3d-44ef-99c7-dad8b761e470 tempest-VolumesSnapshotTestJSON-1750456804 tempest-VolumesSnapshotTestJSON-1750456804-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:27:19.845985 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-4a754171-5f3d-44ef-99c7-dad8b761e470 tempest-VolumesSnapshotTestJSON-1750456804 tempest-VolumesSnapshotTestJSON-1750456804-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:27:19.873826 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-4a754171-5f3d-44ef-99c7-dad8b761e470 tempest-VolumesSnapshotTestJSON-1750456804 tempest-VolumesSnapshotTestJSON-1750456804-project] [instance: c31c26a8-fe5a-40fb-91a1-7b5251fd0daa] Going to try to terminate instance {{(pid=120864) delete /opt/stack/nova/nova/compute/api.py:2719}} Aug 27 02:27:19.965487 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-4a754171-5f3d-44ef-99c7-dad8b761e470 tempest-VolumesSnapshotTestJSON-1750456804 tempest-VolumesSnapshotTestJSON-1750456804-project] 10.222.0.52 "DELETE /compute/v2.1/servers/c31c26a8-fe5a-40fb-91a1-7b5251fd0daa" status: 204 len: 0 microversion: 2.1 time: 0.133957 Aug 27 02:27:19.966348 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 280/559] 10.222.0.52 () {58 vars in 1239 bytes} [Sat Aug 27 02:27:19 2022] DELETE /compute/v2.1/servers/c31c26a8-fe5a-40fb-91a1-7b5251fd0daa => generated 0 bytes in 136 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Aug 27 02:27:19.976099 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-9920d4e7-db69-49a9-8547-9e274f93b0b0 tempest-VolumesSnapshotTestJSON-1750456804 tempest-VolumesSnapshotTestJSON-1750456804-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:27:19.976877 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-9920d4e7-db69-49a9-8547-9e274f93b0b0 tempest-VolumesSnapshotTestJSON-1750456804 tempest-VolumesSnapshotTestJSON-1750456804-project] [instance: c31c26a8-fe5a-40fb-91a1-7b5251fd0daa] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:27:19.983175 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-9920d4e7-db69-49a9-8547-9e274f93b0b0 tempest-VolumesSnapshotTestJSON-1750456804 tempest-VolumesSnapshotTestJSON-1750456804-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:27:19.983689 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-9920d4e7-db69-49a9-8547-9e274f93b0b0 tempest-VolumesSnapshotTestJSON-1750456804 tempest-VolumesSnapshotTestJSON-1750456804-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:27:19.984114 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-9920d4e7-db69-49a9-8547-9e274f93b0b0 tempest-VolumesSnapshotTestJSON-1750456804 tempest-VolumesSnapshotTestJSON-1750456804-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:27:19.984803 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-9920d4e7-db69-49a9-8547-9e274f93b0b0 tempest-VolumesSnapshotTestJSON-1750456804 tempest-VolumesSnapshotTestJSON-1750456804-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:27:19.985306 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-9920d4e7-db69-49a9-8547-9e274f93b0b0 tempest-VolumesSnapshotTestJSON-1750456804 tempest-VolumesSnapshotTestJSON-1750456804-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:27:19.985722 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-9920d4e7-db69-49a9-8547-9e274f93b0b0 tempest-VolumesSnapshotTestJSON-1750456804 tempest-VolumesSnapshotTestJSON-1750456804-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:27:20.141337 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-9920d4e7-db69-49a9-8547-9e274f93b0b0 tempest-VolumesSnapshotTestJSON-1750456804 tempest-VolumesSnapshotTestJSON-1750456804-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '75744eccc3d24d4c9282cbe3a3314489', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fa43d54225bf44b283f9db5ddbbb13ca', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:27:20.150772 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-9920d4e7-db69-49a9-8547-9e274f93b0b0 tempest-VolumesSnapshotTestJSON-1750456804 tempest-VolumesSnapshotTestJSON-1750456804-project] 10.222.0.52 "GET /compute/v2.1/servers/c31c26a8-fe5a-40fb-91a1-7b5251fd0daa" status: 200 len: 1514 microversion: 2.1 time: 0.179630 Aug 27 02:27:20.151373 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 280/560] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:27:19 2022] GET /compute/v2.1/servers/c31c26a8-fe5a-40fb-91a1-7b5251fd0daa => generated 1514 bytes in 181 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:27:21.173181 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-cc6d61c3-1bd9-4421-adf0-e140954df0b9 tempest-VolumesSnapshotTestJSON-1750456804 tempest-VolumesSnapshotTestJSON-1750456804-project] Calling method '>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:27:21.173880 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-cc6d61c3-1bd9-4421-adf0-e140954df0b9 tempest-VolumesSnapshotTestJSON-1750456804 tempest-VolumesSnapshotTestJSON-1750456804-project] [instance: c31c26a8-fe5a-40fb-91a1-7b5251fd0daa] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:27:21.181995 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-cc6d61c3-1bd9-4421-adf0-e140954df0b9 tempest-VolumesSnapshotTestJSON-1750456804 tempest-VolumesSnapshotTestJSON-1750456804-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:27:21.182485 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-cc6d61c3-1bd9-4421-adf0-e140954df0b9 tempest-VolumesSnapshotTestJSON-1750456804 tempest-VolumesSnapshotTestJSON-1750456804-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:27:21.182921 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-cc6d61c3-1bd9-4421-adf0-e140954df0b9 tempest-VolumesSnapshotTestJSON-1750456804 tempest-VolumesSnapshotTestJSON-1750456804-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:27:21.183631 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-cc6d61c3-1bd9-4421-adf0-e140954df0b9 tempest-VolumesSnapshotTestJSON-1750456804 tempest-VolumesSnapshotTestJSON-1750456804-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:27:21.184080 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-cc6d61c3-1bd9-4421-adf0-e140954df0b9 tempest-VolumesSnapshotTestJSON-1750456804 tempest-VolumesSnapshotTestJSON-1750456804-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:27:21.184499 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-cc6d61c3-1bd9-4421-adf0-e140954df0b9 tempest-VolumesSnapshotTestJSON-1750456804 tempest-VolumesSnapshotTestJSON-1750456804-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:27:21.343174 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-cc6d61c3-1bd9-4421-adf0-e140954df0b9 tempest-VolumesSnapshotTestJSON-1750456804 tempest-VolumesSnapshotTestJSON-1750456804-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '75744eccc3d24d4c9282cbe3a3314489', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fa43d54225bf44b283f9db5ddbbb13ca', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:27:21.351767 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-cc6d61c3-1bd9-4421-adf0-e140954df0b9 tempest-VolumesSnapshotTestJSON-1750456804 tempest-VolumesSnapshotTestJSON-1750456804-project] 10.222.0.52 "GET /compute/v2.1/servers/c31c26a8-fe5a-40fb-91a1-7b5251fd0daa" status: 200 len: 1514 microversion: 2.1 time: 0.182827 Aug 27 02:27:21.352339 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 281/561] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:27:21 2022] GET /compute/v2.1/servers/c31c26a8-fe5a-40fb-91a1-7b5251fd0daa => generated 1514 bytes in 184 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:27:22.374620 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-f39ee8f6-32fa-403c-80cf-c6f1a98191e7 tempest-VolumesSnapshotTestJSON-1750456804 tempest-VolumesSnapshotTestJSON-1750456804-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:27:22.375333 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-f39ee8f6-32fa-403c-80cf-c6f1a98191e7 tempest-VolumesSnapshotTestJSON-1750456804 tempest-VolumesSnapshotTestJSON-1750456804-project] [instance: c31c26a8-fe5a-40fb-91a1-7b5251fd0daa] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:27:22.383123 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-f39ee8f6-32fa-403c-80cf-c6f1a98191e7 tempest-VolumesSnapshotTestJSON-1750456804 tempest-VolumesSnapshotTestJSON-1750456804-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:27:22.383622 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-f39ee8f6-32fa-403c-80cf-c6f1a98191e7 tempest-VolumesSnapshotTestJSON-1750456804 tempest-VolumesSnapshotTestJSON-1750456804-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:27:22.384073 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-f39ee8f6-32fa-403c-80cf-c6f1a98191e7 tempest-VolumesSnapshotTestJSON-1750456804 tempest-VolumesSnapshotTestJSON-1750456804-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:27:22.384793 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-f39ee8f6-32fa-403c-80cf-c6f1a98191e7 tempest-VolumesSnapshotTestJSON-1750456804 tempest-VolumesSnapshotTestJSON-1750456804-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:27:22.385362 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-f39ee8f6-32fa-403c-80cf-c6f1a98191e7 tempest-VolumesSnapshotTestJSON-1750456804 tempest-VolumesSnapshotTestJSON-1750456804-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:27:22.385781 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-f39ee8f6-32fa-403c-80cf-c6f1a98191e7 tempest-VolumesSnapshotTestJSON-1750456804 tempest-VolumesSnapshotTestJSON-1750456804-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:27:22.542568 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-f39ee8f6-32fa-403c-80cf-c6f1a98191e7 tempest-VolumesSnapshotTestJSON-1750456804 tempest-VolumesSnapshotTestJSON-1750456804-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '75744eccc3d24d4c9282cbe3a3314489', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fa43d54225bf44b283f9db5ddbbb13ca', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:27:22.547587 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-f39ee8f6-32fa-403c-80cf-c6f1a98191e7 tempest-VolumesSnapshotTestJSON-1750456804 tempest-VolumesSnapshotTestJSON-1750456804-project] 10.222.0.52 "GET /compute/v2.1/servers/c31c26a8-fe5a-40fb-91a1-7b5251fd0daa" status: 200 len: 1514 microversion: 2.1 time: 0.177792 Aug 27 02:27:22.548170 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 281/562] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:27:22 2022] GET /compute/v2.1/servers/c31c26a8-fe5a-40fb-91a1-7b5251fd0daa => generated 1514 bytes in 179 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:27:23.101084 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [req-816657ba-b5cd-4597-bb2f-48220357677f req-e2d90cbb-187f-4907-88a8-e6281ee338d3 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "c31c26a8-fe5a-40fb-91a1-7b5251fd0daa", "name": "network-vif-deleted", "tag": "22831215-7029-41bf-a558-6aa1056c1015"}]} {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 27 02:27:23.117528 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [req-816657ba-b5cd-4597-bb2f-48220357677f req-e2d90cbb-187f-4907-88a8-e6281ee338d3 service nova] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:27:23.118001 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [req-816657ba-b5cd-4597-bb2f-48220357677f req-e2d90cbb-187f-4907-88a8-e6281ee338d3 service nova] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:27:23.118420 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [req-816657ba-b5cd-4597-bb2f-48220357677f req-e2d90cbb-187f-4907-88a8-e6281ee338d3 service nova] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:27:23.130884 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.compute.server_external_events [req-816657ba-b5cd-4597-bb2f-48220357677f req-e2d90cbb-187f-4907-88a8-e6281ee338d3 service nova] Creating event network-vif-deleted:22831215-7029-41bf-a558-6aa1056c1015 for instance c31c26a8-fe5a-40fb-91a1-7b5251fd0daa on hv-cinder-84630 Aug 27 02:27:23.135959 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [req-816657ba-b5cd-4597-bb2f-48220357677f req-e2d90cbb-187f-4907-88a8-e6281ee338d3 service nova] 10.222.0.52 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.043642 Aug 27 02:27:23.137033 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 282/563] 10.222.0.52 () {64 vars in 1317 bytes} [Sat Aug 27 02:27:23 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 27 02:27:23.569756 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-1853d1fe-297e-4f2b-8f49-daa0394e604f tempest-VolumesSnapshotTestJSON-1750456804 tempest-VolumesSnapshotTestJSON-1750456804-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:27:23.570370 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-1853d1fe-297e-4f2b-8f49-daa0394e604f tempest-VolumesSnapshotTestJSON-1750456804 tempest-VolumesSnapshotTestJSON-1750456804-project] [instance: c31c26a8-fe5a-40fb-91a1-7b5251fd0daa] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:27:23.576926 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-1853d1fe-297e-4f2b-8f49-daa0394e604f tempest-VolumesSnapshotTestJSON-1750456804 tempest-VolumesSnapshotTestJSON-1750456804-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:27:23.577451 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-1853d1fe-297e-4f2b-8f49-daa0394e604f tempest-VolumesSnapshotTestJSON-1750456804 tempest-VolumesSnapshotTestJSON-1750456804-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:27:23.577889 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-1853d1fe-297e-4f2b-8f49-daa0394e604f tempest-VolumesSnapshotTestJSON-1750456804 tempest-VolumesSnapshotTestJSON-1750456804-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:27:23.578591 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-1853d1fe-297e-4f2b-8f49-daa0394e604f tempest-VolumesSnapshotTestJSON-1750456804 tempest-VolumesSnapshotTestJSON-1750456804-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:27:23.579349 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-1853d1fe-297e-4f2b-8f49-daa0394e604f tempest-VolumesSnapshotTestJSON-1750456804 tempest-VolumesSnapshotTestJSON-1750456804-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:27:23.579790 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-1853d1fe-297e-4f2b-8f49-daa0394e604f tempest-VolumesSnapshotTestJSON-1750456804 tempest-VolumesSnapshotTestJSON-1750456804-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:27:23.612276 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-1853d1fe-297e-4f2b-8f49-daa0394e604f tempest-VolumesSnapshotTestJSON-1750456804 tempest-VolumesSnapshotTestJSON-1750456804-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:27:23.612661 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-1853d1fe-297e-4f2b-8f49-daa0394e604f tempest-VolumesSnapshotTestJSON-1750456804 tempest-VolumesSnapshotTestJSON-1750456804-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:27:23.613095 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-1853d1fe-297e-4f2b-8f49-daa0394e604f tempest-VolumesSnapshotTestJSON-1750456804 tempest-VolumesSnapshotTestJSON-1750456804-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:27:23.613576 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.objects.instance [None req-1853d1fe-297e-4f2b-8f49-daa0394e604f tempest-VolumesSnapshotTestJSON-1750456804 tempest-VolumesSnapshotTestJSON-1750456804-project] Lazy-loading 'fault' on Instance uuid c31c26a8-fe5a-40fb-91a1-7b5251fd0daa {{(pid=120863) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Aug 27 02:27:23.655725 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-1853d1fe-297e-4f2b-8f49-daa0394e604f tempest-VolumesSnapshotTestJSON-1750456804 tempest-VolumesSnapshotTestJSON-1750456804-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '75744eccc3d24d4c9282cbe3a3314489', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fa43d54225bf44b283f9db5ddbbb13ca', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:27:23.664054 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-1853d1fe-297e-4f2b-8f49-daa0394e604f tempest-VolumesSnapshotTestJSON-1750456804 tempest-VolumesSnapshotTestJSON-1750456804-project] 10.222.0.52 "GET /compute/v2.1/servers/c31c26a8-fe5a-40fb-91a1-7b5251fd0daa" status: 200 len: 1309 microversion: 2.1 time: 0.098220 Aug 27 02:27:23.664646 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 282/564] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:27:23 2022] GET /compute/v2.1/servers/c31c26a8-fe5a-40fb-91a1-7b5251fd0daa => generated 1309 bytes in 100 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:27:24.686252 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-81407f9c-3267-4c80-930c-20cf0c43b120 tempest-VolumesSnapshotTestJSON-1750456804 tempest-VolumesSnapshotTestJSON-1750456804-project] Calling method '>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:27:24.686986 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-81407f9c-3267-4c80-930c-20cf0c43b120 tempest-VolumesSnapshotTestJSON-1750456804 tempest-VolumesSnapshotTestJSON-1750456804-project] [instance: c31c26a8-fe5a-40fb-91a1-7b5251fd0daa] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:27:24.693599 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-81407f9c-3267-4c80-930c-20cf0c43b120 tempest-VolumesSnapshotTestJSON-1750456804 tempest-VolumesSnapshotTestJSON-1750456804-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:27:24.694085 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-81407f9c-3267-4c80-930c-20cf0c43b120 tempest-VolumesSnapshotTestJSON-1750456804 tempest-VolumesSnapshotTestJSON-1750456804-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:27:24.694520 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-81407f9c-3267-4c80-930c-20cf0c43b120 tempest-VolumesSnapshotTestJSON-1750456804 tempest-VolumesSnapshotTestJSON-1750456804-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:27:24.695233 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-81407f9c-3267-4c80-930c-20cf0c43b120 tempest-VolumesSnapshotTestJSON-1750456804 tempest-VolumesSnapshotTestJSON-1750456804-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:27:24.695684 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-81407f9c-3267-4c80-930c-20cf0c43b120 tempest-VolumesSnapshotTestJSON-1750456804 tempest-VolumesSnapshotTestJSON-1750456804-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:27:24.696105 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-81407f9c-3267-4c80-930c-20cf0c43b120 tempest-VolumesSnapshotTestJSON-1750456804 tempest-VolumesSnapshotTestJSON-1750456804-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:27:24.711648 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.wsgi [None req-81407f9c-3267-4c80-930c-20cf0c43b120 tempest-VolumesSnapshotTestJSON-1750456804 tempest-VolumesSnapshotTestJSON-1750456804-project] HTTP exception thrown: Instance c31c26a8-fe5a-40fb-91a1-7b5251fd0daa could not be found. Aug 27 02:27:24.713344 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-81407f9c-3267-4c80-930c-20cf0c43b120 tempest-VolumesSnapshotTestJSON-1750456804 tempest-VolumesSnapshotTestJSON-1750456804-project] Request method failure captured: Aug 27 02:27:24.713344 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: request: GET /compute/v2.1/servers/c31c26a8-fe5a-40fb-91a1-7b5251fd0daa HTTP/1.1 Aug 27 02:27:24.713344 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: Accept: application/json Aug 27 02:27:24.713344 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: Accept-Encoding: identity Aug 27 02:27:24.713344 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: Connection: close Aug 27 02:27:24.713344 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: Content-Length: 0 Aug 27 02:27:24.713344 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: Content-Type: application/json Aug 27 02:27:24.713344 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: Host: 10.222.0.52 Aug 27 02:27:24.713344 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: Openstack-System-Scope: None Aug 27 02:27:24.713344 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: User-Agent: python-urllib3/1.26.11 Aug 27 02:27:24.713344 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-Auth-Token: gAAAAABjCYCjxBX_5JsN8nujRuIFBM1l5sxNDqa6fSs541Cu9nuaAmtZAqYD0aWFW_HnYB0B1tpJZv9bhJBEdRpWrr2ex3FbPNvfD0J3cVNqea_fdQysA0iZ33xjIH6pT-txE713G8wEM_gGr6-7jzRmTTMULp5yqwF_b7x9d1RaH_UV7L3KktA Aug 27 02:27:24.713344 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-Domain-Id: None Aug 27 02:27:24.713344 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-Domain-Name: None Aug 27 02:27:24.713344 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-Identity-Status: Confirmed Aug 27 02:27:24.713344 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-Is-Admin-Project: True Aug 27 02:27:24.713344 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-Project-Domain-Id: default Aug 27 02:27:24.713344 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-Project-Domain-Name: Default Aug 27 02:27:24.713344 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-Project-Id: fa43d54225bf44b283f9db5ddbbb13ca Aug 27 02:27:24.713344 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-Project-Name: tempest-VolumesSnapshotTestJSON-1750456804 Aug 27 02:27:24.713344 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-Role: reader,member Aug 27 02:27:24.713344 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-Roles: reader,member Aug 27 02:27:24.714730 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-Service-Catalog: [{"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.52:8080", "publicURL": "http://10.222.0.52:8080/v1/AUTH_fa43d54225bf44b283f9db5ddbbb13ca"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52:9696/networking"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/compute/v2/fa43d54225bf44b283f9db5ddbbb13ca"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/image"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/volume/v3/fa43d54225bf44b283f9db5ddbbb13ca"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/identity"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/volume/v3/fa43d54225bf44b283f9db5ddbbb13ca"}]}] Aug 27 02:27:24.714730 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-Tenant: tempest-VolumesSnapshotTestJSON-1750456804 Aug 27 02:27:24.714730 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-Tenant-Id: fa43d54225bf44b283f9db5ddbbb13ca Aug 27 02:27:24.714730 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-Tenant-Name: tempest-VolumesSnapshotTestJSON-1750456804 Aug 27 02:27:24.714730 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-User: tempest-VolumesSnapshotTestJSON-1750456804-project Aug 27 02:27:24.714730 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-User-Domain-Id: default Aug 27 02:27:24.714730 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-User-Domain-Name: Default Aug 27 02:27:24.714730 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-User-Id: 75744eccc3d24d4c9282cbe3a3314489 Aug 27 02:27:24.714730 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-User-Name: tempest-VolumesSnapshotTestJSON-1750456804-project Aug 27 02:27:24.714730 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: method: > Aug 27 02:27:24.714730 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: exception: Instance c31c26a8-fe5a-40fb-91a1-7b5251fd0daa could not be found. Aug 27 02:27:24.714730 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: action_args: {'id': 'c31c26a8-fe5a-40fb-91a1-7b5251fd0daa'} Aug 27 02:27:24.715603 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Aug 27 02:27:24.715603 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 27 02:27:24.715603 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Aug 27 02:27:24.715603 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Aug 27 02:27:24.715603 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Aug 27 02:27:24.715603 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Aug 27 02:27:24.715603 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Aug 27 02:27:24.715603 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Aug 27 02:27:24.715603 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Aug 27 02:27:24.715603 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Aug 27 02:27:24.715603 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance c31c26a8-fe5a-40fb-91a1-7b5251fd0daa could not be found. Aug 27 02:27:24.715603 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi Aug 27 02:27:24.715603 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 27 02:27:24.715603 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi Aug 27 02:27:24.715603 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 27 02:27:24.715603 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 27 02:27:24.715603 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 27 02:27:24.715603 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Aug 27 02:27:24.715603 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Aug 27 02:27:24.715603 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Aug 27 02:27:24.718366 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Aug 27 02:27:24.718366 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Aug 27 02:27:24.718366 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi instance = self._get_server( Aug 27 02:27:24.718366 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Aug 27 02:27:24.718366 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Aug 27 02:27:24.718366 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Aug 27 02:27:24.718366 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Aug 27 02:27:24.718366 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance c31c26a8-fe5a-40fb-91a1-7b5251fd0daa could not be found. Aug 27 02:27:24.718366 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi Aug 27 02:27:24.718366 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 27 02:27:24.718366 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi Aug 27 02:27:24.718366 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 27 02:27:24.718366 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 27 02:27:24.718366 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 27 02:27:24.718366 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Aug 27 02:27:24.718366 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Aug 27 02:27:24.718366 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance c31c26a8-fe5a-40fb-91a1-7b5251fd0daa could not be found. Aug 27 02:27:24.718366 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi Aug 27 02:27:24.720889 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-81407f9c-3267-4c80-930c-20cf0c43b120 tempest-VolumesSnapshotTestJSON-1750456804 tempest-VolumesSnapshotTestJSON-1750456804-project] Returning 404 to user: Instance c31c26a8-fe5a-40fb-91a1-7b5251fd0daa could not be found. {{(pid=120864) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Aug 27 02:27:24.720889 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-81407f9c-3267-4c80-930c-20cf0c43b120 tempest-VolumesSnapshotTestJSON-1750456804 tempest-VolumesSnapshotTestJSON-1750456804-project] 10.222.0.52 "GET /compute/v2.1/servers/c31c26a8-fe5a-40fb-91a1-7b5251fd0daa" status: 404 len: 111 microversion: 2.1 time: 0.033460 Aug 27 02:27:24.720889 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 283/565] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:27:24 2022] GET /compute/v2.1/servers/c31c26a8-fe5a-40fb-91a1-7b5251fd0daa => generated 111 bytes in 35 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Aug 27 02:27:41.968088 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-18490069-b68d-4880-9abf-b3e8cfa75c11 tempest-VolumesSnapshotTestJSON-1750456804 tempest-VolumesSnapshotTestJSON-1750456804-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-VolumesSnapshotTestJSON-instance-687003294", "imageRef": "9fc30b36-b827-46f9-892d-30e808bc2b8a", "flavorRef": "452", "networks": [{"uuid": "c0b61970-d693-42b1-b7a6-dd3cf1b79e9d"}]}} {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 27 02:27:41.977401 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG dbcounter [-] [120863] Writing DB stats nova_api:SELECT=23 {{(pid=120863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 27 02:27:41.978177 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG dbcounter [-] [120863] Writing DB stats nova_cell1:SELECT=31 {{(pid=120863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 27 02:27:42.194135 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.network.neutron [None req-18490069-b68d-4880-9abf-b3e8cfa75c11 tempest-VolumesSnapshotTestJSON-1750456804 tempest-VolumesSnapshotTestJSON-1750456804-project] validate_networks() for [('c0b61970-d693-42b1-b7a6-dd3cf1b79e9d', None, None, None, None, None)] {{(pid=120863) validate_networks /opt/stack/nova/nova/network/neutron.py:2626}} Aug 27 02:27:42.654615 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.virt.hardware [None req-18490069-b68d-4880-9abf-b3e8cfa75c11 tempest-VolumesSnapshotTestJSON-1750456804 tempest-VolumesSnapshotTestJSON-1750456804-project] Flavor limits 0:0:0 {{(pid=120863) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Aug 27 02:27:42.655143 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.virt.hardware [None req-18490069-b68d-4880-9abf-b3e8cfa75c11 tempest-VolumesSnapshotTestJSON-1750456804 tempest-VolumesSnapshotTestJSON-1750456804-project] Image limits 0:0:0 {{(pid=120863) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Aug 27 02:27:42.655647 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.virt.hardware [None req-18490069-b68d-4880-9abf-b3e8cfa75c11 tempest-VolumesSnapshotTestJSON-1750456804 tempest-VolumesSnapshotTestJSON-1750456804-project] Flavor pref 0:0:0 {{(pid=120863) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Aug 27 02:27:42.656266 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.virt.hardware [None req-18490069-b68d-4880-9abf-b3e8cfa75c11 tempest-VolumesSnapshotTestJSON-1750456804 tempest-VolumesSnapshotTestJSON-1750456804-project] Image pref 0:0:0 {{(pid=120863) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Aug 27 02:27:42.656725 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.virt.hardware [None req-18490069-b68d-4880-9abf-b3e8cfa75c11 tempest-VolumesSnapshotTestJSON-1750456804 tempest-VolumesSnapshotTestJSON-1750456804-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=120863) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Aug 27 02:27:42.667466 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-18490069-b68d-4880-9abf-b3e8cfa75c11 tempest-VolumesSnapshotTestJSON-1750456804 tempest-VolumesSnapshotTestJSON-1750456804-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:27:42.668065 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-18490069-b68d-4880-9abf-b3e8cfa75c11 tempest-VolumesSnapshotTestJSON-1750456804 tempest-VolumesSnapshotTestJSON-1750456804-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:27:42.668493 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-18490069-b68d-4880-9abf-b3e8cfa75c11 tempest-VolumesSnapshotTestJSON-1750456804 tempest-VolumesSnapshotTestJSON-1750456804-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:27:42.669243 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-18490069-b68d-4880-9abf-b3e8cfa75c11 tempest-VolumesSnapshotTestJSON-1750456804 tempest-VolumesSnapshotTestJSON-1750456804-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:27:42.669693 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-18490069-b68d-4880-9abf-b3e8cfa75c11 tempest-VolumesSnapshotTestJSON-1750456804 tempest-VolumesSnapshotTestJSON-1750456804-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:27:42.670164 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-18490069-b68d-4880-9abf-b3e8cfa75c11 tempest-VolumesSnapshotTestJSON-1750456804 tempest-VolumesSnapshotTestJSON-1750456804-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:27:42.688965 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.quota [None req-18490069-b68d-4880-9abf-b3e8cfa75c11 tempest-VolumesSnapshotTestJSON-1750456804 tempest-VolumesSnapshotTestJSON-1750456804-project] Getting quotas for project fa43d54225bf44b283f9db5ddbbb13ca. Resources: {'cores', 'ram', 'instances'} {{(pid=120863) _get_quotas /opt/stack/nova/nova/quota.py:393}} Aug 27 02:27:42.693146 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.quota [None req-18490069-b68d-4880-9abf-b3e8cfa75c11 tempest-VolumesSnapshotTestJSON-1750456804 tempest-VolumesSnapshotTestJSON-1750456804-project] Getting quotas for user 75744eccc3d24d4c9282cbe3a3314489 and project fa43d54225bf44b283f9db5ddbbb13ca. Resources: {'cores', 'ram', 'instances'} {{(pid=120863) _get_quotas /opt/stack/nova/nova/quota.py:383}} Aug 27 02:27:42.705531 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-18490069-b68d-4880-9abf-b3e8cfa75c11 tempest-VolumesSnapshotTestJSON-1750456804 tempest-VolumesSnapshotTestJSON-1750456804-project] Going to run 1 instances... {{(pid=120863) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Aug 27 02:27:42.707952 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-18490069-b68d-4880-9abf-b3e8cfa75c11 tempest-VolumesSnapshotTestJSON-1750456804 tempest-VolumesSnapshotTestJSON-1750456804-project] [instance: 46dae88e-3429-47c7-9376-c4c0d9cddd29] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='9fc30b36-b827-46f9-892d-30e808bc2b8a',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=120863) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Aug 27 02:27:42.755231 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-18490069-b68d-4880-9abf-b3e8cfa75c11 tempest-VolumesSnapshotTestJSON-1750456804 tempest-VolumesSnapshotTestJSON-1750456804-project] 10.222.0.52 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 0.790747 Aug 27 02:27:42.756185 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 283/566] 10.222.0.52 () {60 vars in 1147 bytes} [Sat Aug 27 02:27:41 2022] POST /compute/v2.1/servers => generated 380 bytes in 793 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Aug 27 02:27:42.770043 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-58f1c94a-f0fc-49f4-921f-4ce6acdd9a74 tempest-VolumesSnapshotTestJSON-1750456804 tempest-VolumesSnapshotTestJSON-1750456804-project] Calling method '>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:27:42.771173 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-58f1c94a-f0fc-49f4-921f-4ce6acdd9a74 tempest-VolumesSnapshotTestJSON-1750456804 tempest-VolumesSnapshotTestJSON-1750456804-project] [instance: 46dae88e-3429-47c7-9376-c4c0d9cddd29] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:27:42.776363 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG dbcounter [-] [120864] Writing DB stats nova_api:SELECT=30,nova_api:UPDATE=1 {{(pid=120864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 27 02:27:42.778903 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG dbcounter [-] [120864] Writing DB stats nova_cell1:SELECT=80,nova_cell1:INSERT=2,nova_cell1:UPDATE=1 {{(pid=120864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 27 02:27:42.897050 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-58f1c94a-f0fc-49f4-921f-4ce6acdd9a74 tempest-VolumesSnapshotTestJSON-1750456804 tempest-VolumesSnapshotTestJSON-1750456804-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '75744eccc3d24d4c9282cbe3a3314489', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fa43d54225bf44b283f9db5ddbbb13ca', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:27:42.904546 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-58f1c94a-f0fc-49f4-921f-4ce6acdd9a74 tempest-VolumesSnapshotTestJSON-1750456804 tempest-VolumesSnapshotTestJSON-1750456804-project] 10.222.0.52 "GET /compute/v2.1/servers/46dae88e-3429-47c7-9376-c4c0d9cddd29" status: 200 len: 1218 microversion: 2.1 time: 0.138981 Aug 27 02:27:42.917171 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 284/567] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:27:42 2022] GET /compute/v2.1/servers/46dae88e-3429-47c7-9376-c4c0d9cddd29 => generated 1218 bytes in 141 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:27:43.930310 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-c26595e2-5c7b-433b-8629-563038b81817 tempest-VolumesSnapshotTestJSON-1750456804 tempest-VolumesSnapshotTestJSON-1750456804-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:27:43.930878 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-c26595e2-5c7b-433b-8629-563038b81817 tempest-VolumesSnapshotTestJSON-1750456804 tempest-VolumesSnapshotTestJSON-1750456804-project] [instance: 46dae88e-3429-47c7-9376-c4c0d9cddd29] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:27:43.941435 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-c26595e2-5c7b-433b-8629-563038b81817 tempest-VolumesSnapshotTestJSON-1750456804 tempest-VolumesSnapshotTestJSON-1750456804-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:27:43.941916 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-c26595e2-5c7b-433b-8629-563038b81817 tempest-VolumesSnapshotTestJSON-1750456804 tempest-VolumesSnapshotTestJSON-1750456804-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:27:43.942372 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-c26595e2-5c7b-433b-8629-563038b81817 tempest-VolumesSnapshotTestJSON-1750456804 tempest-VolumesSnapshotTestJSON-1750456804-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:27:43.943176 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-c26595e2-5c7b-433b-8629-563038b81817 tempest-VolumesSnapshotTestJSON-1750456804 tempest-VolumesSnapshotTestJSON-1750456804-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:27:43.943644 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-c26595e2-5c7b-433b-8629-563038b81817 tempest-VolumesSnapshotTestJSON-1750456804 tempest-VolumesSnapshotTestJSON-1750456804-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:27:43.944058 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-c26595e2-5c7b-433b-8629-563038b81817 tempest-VolumesSnapshotTestJSON-1750456804 tempest-VolumesSnapshotTestJSON-1750456804-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:27:44.013878 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-c26595e2-5c7b-433b-8629-563038b81817 tempest-VolumesSnapshotTestJSON-1750456804 tempest-VolumesSnapshotTestJSON-1750456804-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '75744eccc3d24d4c9282cbe3a3314489', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fa43d54225bf44b283f9db5ddbbb13ca', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:27:44.026537 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-c26595e2-5c7b-433b-8629-563038b81817 tempest-VolumesSnapshotTestJSON-1750456804 tempest-VolumesSnapshotTestJSON-1750456804-project] 10.222.0.52 "GET /compute/v2.1/servers/46dae88e-3429-47c7-9376-c4c0d9cddd29" status: 200 len: 1270 microversion: 2.1 time: 0.103672 Aug 27 02:27:44.027781 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 284/568] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:27:43 2022] GET /compute/v2.1/servers/46dae88e-3429-47c7-9376-c4c0d9cddd29 => generated 1270 bytes in 106 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:27:45.059692 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-bce6fcaa-644f-4749-a915-4fd4ea3001d6 tempest-VolumesSnapshotTestJSON-1750456804 tempest-VolumesSnapshotTestJSON-1750456804-project] Calling method '>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:27:45.060269 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-bce6fcaa-644f-4749-a915-4fd4ea3001d6 tempest-VolumesSnapshotTestJSON-1750456804 tempest-VolumesSnapshotTestJSON-1750456804-project] [instance: 46dae88e-3429-47c7-9376-c4c0d9cddd29] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:27:45.069131 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-bce6fcaa-644f-4749-a915-4fd4ea3001d6 tempest-VolumesSnapshotTestJSON-1750456804 tempest-VolumesSnapshotTestJSON-1750456804-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:27:45.069725 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-bce6fcaa-644f-4749-a915-4fd4ea3001d6 tempest-VolumesSnapshotTestJSON-1750456804 tempest-VolumesSnapshotTestJSON-1750456804-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:27:45.070193 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-bce6fcaa-644f-4749-a915-4fd4ea3001d6 tempest-VolumesSnapshotTestJSON-1750456804 tempest-VolumesSnapshotTestJSON-1750456804-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:27:45.071104 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-bce6fcaa-644f-4749-a915-4fd4ea3001d6 tempest-VolumesSnapshotTestJSON-1750456804 tempest-VolumesSnapshotTestJSON-1750456804-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:27:45.071598 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-bce6fcaa-644f-4749-a915-4fd4ea3001d6 tempest-VolumesSnapshotTestJSON-1750456804 tempest-VolumesSnapshotTestJSON-1750456804-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:27:45.072030 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-bce6fcaa-644f-4749-a915-4fd4ea3001d6 tempest-VolumesSnapshotTestJSON-1750456804 tempest-VolumesSnapshotTestJSON-1750456804-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:27:45.221156 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-bce6fcaa-644f-4749-a915-4fd4ea3001d6 tempest-VolumesSnapshotTestJSON-1750456804 tempest-VolumesSnapshotTestJSON-1750456804-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '75744eccc3d24d4c9282cbe3a3314489', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fa43d54225bf44b283f9db5ddbbb13ca', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:27:45.229292 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-bce6fcaa-644f-4749-a915-4fd4ea3001d6 tempest-VolumesSnapshotTestJSON-1750456804 tempest-VolumesSnapshotTestJSON-1750456804-project] 10.222.0.52 "GET /compute/v2.1/servers/46dae88e-3429-47c7-9376-c4c0d9cddd29" status: 200 len: 1318 microversion: 2.1 time: 0.173341 Aug 27 02:27:45.230064 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 285/569] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:27:45 2022] GET /compute/v2.1/servers/46dae88e-3429-47c7-9376-c4c0d9cddd29 => generated 1318 bytes in 175 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:27:45.748486 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [req-94cb5c6f-a1e4-4d3c-b512-d0e4c5950ead req-01371196-1ff1-4f97-8b60-c9ca038da82f service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "46dae88e-3429-47c7-9376-c4c0d9cddd29", "tag": "692449b2-a058-408c-bf04-f924a65e0f5a"}]} {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 27 02:27:45.759938 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [req-94cb5c6f-a1e4-4d3c-b512-d0e4c5950ead req-01371196-1ff1-4f97-8b60-c9ca038da82f service nova] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:27:45.760793 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [req-94cb5c6f-a1e4-4d3c-b512-d0e4c5950ead req-01371196-1ff1-4f97-8b60-c9ca038da82f service nova] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:27:45.761397 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [req-94cb5c6f-a1e4-4d3c-b512-d0e4c5950ead req-01371196-1ff1-4f97-8b60-c9ca038da82f service nova] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:27:45.777647 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.compute.server_external_events [req-94cb5c6f-a1e4-4d3c-b512-d0e4c5950ead req-01371196-1ff1-4f97-8b60-c9ca038da82f service nova] Creating event network-changed:692449b2-a058-408c-bf04-f924a65e0f5a for instance 46dae88e-3429-47c7-9376-c4c0d9cddd29 on hv-cinder-84630 Aug 27 02:27:45.784977 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [req-94cb5c6f-a1e4-4d3c-b512-d0e4c5950ead req-01371196-1ff1-4f97-8b60-c9ca038da82f service nova] 10.222.0.52 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.040070 Aug 27 02:27:45.785751 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 285/570] 10.222.0.52 () {64 vars in 1317 bytes} [Sat Aug 27 02:27:45 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 42 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Aug 27 02:27:46.251015 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-8128466e-63ee-471e-906b-d7748b12efa8 tempest-VolumesSnapshotTestJSON-1750456804 tempest-VolumesSnapshotTestJSON-1750456804-project] Calling method '>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:27:46.251609 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-8128466e-63ee-471e-906b-d7748b12efa8 tempest-VolumesSnapshotTestJSON-1750456804 tempest-VolumesSnapshotTestJSON-1750456804-project] [instance: 46dae88e-3429-47c7-9376-c4c0d9cddd29] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:27:46.259317 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-8128466e-63ee-471e-906b-d7748b12efa8 tempest-VolumesSnapshotTestJSON-1750456804 tempest-VolumesSnapshotTestJSON-1750456804-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:27:46.259928 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-8128466e-63ee-471e-906b-d7748b12efa8 tempest-VolumesSnapshotTestJSON-1750456804 tempest-VolumesSnapshotTestJSON-1750456804-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:27:46.260314 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-8128466e-63ee-471e-906b-d7748b12efa8 tempest-VolumesSnapshotTestJSON-1750456804 tempest-VolumesSnapshotTestJSON-1750456804-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:27:46.261052 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-8128466e-63ee-471e-906b-d7748b12efa8 tempest-VolumesSnapshotTestJSON-1750456804 tempest-VolumesSnapshotTestJSON-1750456804-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:27:46.261527 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-8128466e-63ee-471e-906b-d7748b12efa8 tempest-VolumesSnapshotTestJSON-1750456804 tempest-VolumesSnapshotTestJSON-1750456804-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:27:46.261949 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-8128466e-63ee-471e-906b-d7748b12efa8 tempest-VolumesSnapshotTestJSON-1750456804 tempest-VolumesSnapshotTestJSON-1750456804-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:27:46.401412 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-8128466e-63ee-471e-906b-d7748b12efa8 tempest-VolumesSnapshotTestJSON-1750456804 tempest-VolumesSnapshotTestJSON-1750456804-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '75744eccc3d24d4c9282cbe3a3314489', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fa43d54225bf44b283f9db5ddbbb13ca', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:27:46.409569 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-8128466e-63ee-471e-906b-d7748b12efa8 tempest-VolumesSnapshotTestJSON-1750456804 tempest-VolumesSnapshotTestJSON-1750456804-project] 10.222.0.52 "GET /compute/v2.1/servers/46dae88e-3429-47c7-9376-c4c0d9cddd29" status: 200 len: 1318 microversion: 2.1 time: 0.162576 Aug 27 02:27:46.410113 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 286/571] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:27:46 2022] GET /compute/v2.1/servers/46dae88e-3429-47c7-9376-c4c0d9cddd29 => generated 1318 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:27:47.433484 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-23a052bf-ce9a-4739-b63e-1446e7cd21ae tempest-VolumesSnapshotTestJSON-1750456804 tempest-VolumesSnapshotTestJSON-1750456804-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:27:47.434170 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-23a052bf-ce9a-4739-b63e-1446e7cd21ae tempest-VolumesSnapshotTestJSON-1750456804 tempest-VolumesSnapshotTestJSON-1750456804-project] [instance: 46dae88e-3429-47c7-9376-c4c0d9cddd29] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:27:47.442288 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-23a052bf-ce9a-4739-b63e-1446e7cd21ae tempest-VolumesSnapshotTestJSON-1750456804 tempest-VolumesSnapshotTestJSON-1750456804-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:27:47.443333 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-23a052bf-ce9a-4739-b63e-1446e7cd21ae tempest-VolumesSnapshotTestJSON-1750456804 tempest-VolumesSnapshotTestJSON-1750456804-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:27:47.443766 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-23a052bf-ce9a-4739-b63e-1446e7cd21ae tempest-VolumesSnapshotTestJSON-1750456804 tempest-VolumesSnapshotTestJSON-1750456804-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:27:47.444459 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-23a052bf-ce9a-4739-b63e-1446e7cd21ae tempest-VolumesSnapshotTestJSON-1750456804 tempest-VolumesSnapshotTestJSON-1750456804-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:27:47.444954 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-23a052bf-ce9a-4739-b63e-1446e7cd21ae tempest-VolumesSnapshotTestJSON-1750456804 tempest-VolumesSnapshotTestJSON-1750456804-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:27:47.445577 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-23a052bf-ce9a-4739-b63e-1446e7cd21ae tempest-VolumesSnapshotTestJSON-1750456804 tempest-VolumesSnapshotTestJSON-1750456804-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:27:47.591548 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-23a052bf-ce9a-4739-b63e-1446e7cd21ae tempest-VolumesSnapshotTestJSON-1750456804 tempest-VolumesSnapshotTestJSON-1750456804-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '75744eccc3d24d4c9282cbe3a3314489', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fa43d54225bf44b283f9db5ddbbb13ca', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:27:47.603256 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-23a052bf-ce9a-4739-b63e-1446e7cd21ae tempest-VolumesSnapshotTestJSON-1750456804 tempest-VolumesSnapshotTestJSON-1750456804-project] 10.222.0.52 "GET /compute/v2.1/servers/46dae88e-3429-47c7-9376-c4c0d9cddd29" status: 200 len: 1318 microversion: 2.1 time: 0.175182 Aug 27 02:27:47.606280 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 286/572] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:27:47 2022] GET /compute/v2.1/servers/46dae88e-3429-47c7-9376-c4c0d9cddd29 => generated 1318 bytes in 177 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:27:48.627769 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-03757632-65e7-4774-a890-9c581c4fc784 tempest-VolumesSnapshotTestJSON-1750456804 tempest-VolumesSnapshotTestJSON-1750456804-project] Calling method '>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:27:48.628428 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-03757632-65e7-4774-a890-9c581c4fc784 tempest-VolumesSnapshotTestJSON-1750456804 tempest-VolumesSnapshotTestJSON-1750456804-project] [instance: 46dae88e-3429-47c7-9376-c4c0d9cddd29] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:27:48.635359 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-03757632-65e7-4774-a890-9c581c4fc784 tempest-VolumesSnapshotTestJSON-1750456804 tempest-VolumesSnapshotTestJSON-1750456804-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:27:48.635837 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-03757632-65e7-4774-a890-9c581c4fc784 tempest-VolumesSnapshotTestJSON-1750456804 tempest-VolumesSnapshotTestJSON-1750456804-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:27:48.636261 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-03757632-65e7-4774-a890-9c581c4fc784 tempest-VolumesSnapshotTestJSON-1750456804 tempest-VolumesSnapshotTestJSON-1750456804-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:27:48.636950 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-03757632-65e7-4774-a890-9c581c4fc784 tempest-VolumesSnapshotTestJSON-1750456804 tempest-VolumesSnapshotTestJSON-1750456804-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:27:48.637427 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-03757632-65e7-4774-a890-9c581c4fc784 tempest-VolumesSnapshotTestJSON-1750456804 tempest-VolumesSnapshotTestJSON-1750456804-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:27:48.637834 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-03757632-65e7-4774-a890-9c581c4fc784 tempest-VolumesSnapshotTestJSON-1750456804 tempest-VolumesSnapshotTestJSON-1750456804-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:27:48.772580 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-03757632-65e7-4774-a890-9c581c4fc784 tempest-VolumesSnapshotTestJSON-1750456804 tempest-VolumesSnapshotTestJSON-1750456804-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '75744eccc3d24d4c9282cbe3a3314489', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fa43d54225bf44b283f9db5ddbbb13ca', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:27:48.780450 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-03757632-65e7-4774-a890-9c581c4fc784 tempest-VolumesSnapshotTestJSON-1750456804 tempest-VolumesSnapshotTestJSON-1750456804-project] 10.222.0.52 "GET /compute/v2.1/servers/46dae88e-3429-47c7-9376-c4c0d9cddd29" status: 200 len: 1318 microversion: 2.1 time: 0.157517 Aug 27 02:27:48.781021 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 287/573] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:27:48 2022] GET /compute/v2.1/servers/46dae88e-3429-47c7-9376-c4c0d9cddd29 => generated 1318 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:27:49.804479 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-05a99aea-0a13-4b11-838a-778165f11f54 tempest-VolumesSnapshotTestJSON-1750456804 tempest-VolumesSnapshotTestJSON-1750456804-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:27:49.805077 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-05a99aea-0a13-4b11-838a-778165f11f54 tempest-VolumesSnapshotTestJSON-1750456804 tempest-VolumesSnapshotTestJSON-1750456804-project] [instance: 46dae88e-3429-47c7-9376-c4c0d9cddd29] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:27:49.812563 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-05a99aea-0a13-4b11-838a-778165f11f54 tempest-VolumesSnapshotTestJSON-1750456804 tempest-VolumesSnapshotTestJSON-1750456804-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:27:49.813047 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-05a99aea-0a13-4b11-838a-778165f11f54 tempest-VolumesSnapshotTestJSON-1750456804 tempest-VolumesSnapshotTestJSON-1750456804-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:27:49.813476 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-05a99aea-0a13-4b11-838a-778165f11f54 tempest-VolumesSnapshotTestJSON-1750456804 tempest-VolumesSnapshotTestJSON-1750456804-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:27:49.814213 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-05a99aea-0a13-4b11-838a-778165f11f54 tempest-VolumesSnapshotTestJSON-1750456804 tempest-VolumesSnapshotTestJSON-1750456804-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:27:49.814659 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-05a99aea-0a13-4b11-838a-778165f11f54 tempest-VolumesSnapshotTestJSON-1750456804 tempest-VolumesSnapshotTestJSON-1750456804-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:27:49.815080 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-05a99aea-0a13-4b11-838a-778165f11f54 tempest-VolumesSnapshotTestJSON-1750456804 tempest-VolumesSnapshotTestJSON-1750456804-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:27:49.965297 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-05a99aea-0a13-4b11-838a-778165f11f54 tempest-VolumesSnapshotTestJSON-1750456804 tempest-VolumesSnapshotTestJSON-1750456804-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '75744eccc3d24d4c9282cbe3a3314489', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fa43d54225bf44b283f9db5ddbbb13ca', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:27:49.973450 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-05a99aea-0a13-4b11-838a-778165f11f54 tempest-VolumesSnapshotTestJSON-1750456804 tempest-VolumesSnapshotTestJSON-1750456804-project] 10.222.0.52 "GET /compute/v2.1/servers/46dae88e-3429-47c7-9376-c4c0d9cddd29" status: 200 len: 1318 microversion: 2.1 time: 0.173334 Aug 27 02:27:49.974436 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 287/574] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:27:49 2022] GET /compute/v2.1/servers/46dae88e-3429-47c7-9376-c4c0d9cddd29 => generated 1318 bytes in 175 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:27:50.995041 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-ac816e2c-47df-4e89-b285-b8525e1bf806 tempest-VolumesSnapshotTestJSON-1750456804 tempest-VolumesSnapshotTestJSON-1750456804-project] Calling method '>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:27:50.995743 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-ac816e2c-47df-4e89-b285-b8525e1bf806 tempest-VolumesSnapshotTestJSON-1750456804 tempest-VolumesSnapshotTestJSON-1750456804-project] [instance: 46dae88e-3429-47c7-9376-c4c0d9cddd29] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:27:51.003450 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-ac816e2c-47df-4e89-b285-b8525e1bf806 tempest-VolumesSnapshotTestJSON-1750456804 tempest-VolumesSnapshotTestJSON-1750456804-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:27:51.003960 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-ac816e2c-47df-4e89-b285-b8525e1bf806 tempest-VolumesSnapshotTestJSON-1750456804 tempest-VolumesSnapshotTestJSON-1750456804-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:27:51.004398 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-ac816e2c-47df-4e89-b285-b8525e1bf806 tempest-VolumesSnapshotTestJSON-1750456804 tempest-VolumesSnapshotTestJSON-1750456804-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:27:51.005247 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-ac816e2c-47df-4e89-b285-b8525e1bf806 tempest-VolumesSnapshotTestJSON-1750456804 tempest-VolumesSnapshotTestJSON-1750456804-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:27:51.005700 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-ac816e2c-47df-4e89-b285-b8525e1bf806 tempest-VolumesSnapshotTestJSON-1750456804 tempest-VolumesSnapshotTestJSON-1750456804-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:27:51.006120 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-ac816e2c-47df-4e89-b285-b8525e1bf806 tempest-VolumesSnapshotTestJSON-1750456804 tempest-VolumesSnapshotTestJSON-1750456804-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:27:51.156113 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-ac816e2c-47df-4e89-b285-b8525e1bf806 tempest-VolumesSnapshotTestJSON-1750456804 tempest-VolumesSnapshotTestJSON-1750456804-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '75744eccc3d24d4c9282cbe3a3314489', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fa43d54225bf44b283f9db5ddbbb13ca', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:27:51.169414 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-ac816e2c-47df-4e89-b285-b8525e1bf806 tempest-VolumesSnapshotTestJSON-1750456804 tempest-VolumesSnapshotTestJSON-1750456804-project] 10.222.0.52 "GET /compute/v2.1/servers/46dae88e-3429-47c7-9376-c4c0d9cddd29" status: 200 len: 1318 microversion: 2.1 time: 0.178077 Aug 27 02:27:51.170051 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 288/575] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:27:50 2022] GET /compute/v2.1/servers/46dae88e-3429-47c7-9376-c4c0d9cddd29 => generated 1318 bytes in 180 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:27:52.191575 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-0888fef4-b6b5-4ce9-beed-566466086e33 tempest-VolumesSnapshotTestJSON-1750456804 tempest-VolumesSnapshotTestJSON-1750456804-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:27:52.192222 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-0888fef4-b6b5-4ce9-beed-566466086e33 tempest-VolumesSnapshotTestJSON-1750456804 tempest-VolumesSnapshotTestJSON-1750456804-project] [instance: 46dae88e-3429-47c7-9376-c4c0d9cddd29] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:27:52.201873 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-0888fef4-b6b5-4ce9-beed-566466086e33 tempest-VolumesSnapshotTestJSON-1750456804 tempest-VolumesSnapshotTestJSON-1750456804-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:27:52.202561 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-0888fef4-b6b5-4ce9-beed-566466086e33 tempest-VolumesSnapshotTestJSON-1750456804 tempest-VolumesSnapshotTestJSON-1750456804-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:27:52.203001 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-0888fef4-b6b5-4ce9-beed-566466086e33 tempest-VolumesSnapshotTestJSON-1750456804 tempest-VolumesSnapshotTestJSON-1750456804-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:27:52.204159 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-0888fef4-b6b5-4ce9-beed-566466086e33 tempest-VolumesSnapshotTestJSON-1750456804 tempest-VolumesSnapshotTestJSON-1750456804-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:27:52.205390 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-0888fef4-b6b5-4ce9-beed-566466086e33 tempest-VolumesSnapshotTestJSON-1750456804 tempest-VolumesSnapshotTestJSON-1750456804-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:27:52.206028 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-0888fef4-b6b5-4ce9-beed-566466086e33 tempest-VolumesSnapshotTestJSON-1750456804 tempest-VolumesSnapshotTestJSON-1750456804-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:27:52.354528 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-0888fef4-b6b5-4ce9-beed-566466086e33 tempest-VolumesSnapshotTestJSON-1750456804 tempest-VolumesSnapshotTestJSON-1750456804-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '75744eccc3d24d4c9282cbe3a3314489', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fa43d54225bf44b283f9db5ddbbb13ca', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:27:52.362820 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-0888fef4-b6b5-4ce9-beed-566466086e33 tempest-VolumesSnapshotTestJSON-1750456804 tempest-VolumesSnapshotTestJSON-1750456804-project] 10.222.0.52 "GET /compute/v2.1/servers/46dae88e-3429-47c7-9376-c4c0d9cddd29" status: 200 len: 1318 microversion: 2.1 time: 0.175604 Aug 27 02:27:52.363383 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 288/576] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:27:52 2022] GET /compute/v2.1/servers/46dae88e-3429-47c7-9376-c4c0d9cddd29 => generated 1318 bytes in 177 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:27:53.385665 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-ee3b7a06-09b0-4e42-9446-286b3fb10fde tempest-VolumesSnapshotTestJSON-1750456804 tempest-VolumesSnapshotTestJSON-1750456804-project] Calling method '>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:27:53.386283 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-ee3b7a06-09b0-4e42-9446-286b3fb10fde tempest-VolumesSnapshotTestJSON-1750456804 tempest-VolumesSnapshotTestJSON-1750456804-project] [instance: 46dae88e-3429-47c7-9376-c4c0d9cddd29] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:27:53.389521 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG dbcounter [-] [120864] Writing DB stats nova_cell0:SELECT=2 {{(pid=120864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 27 02:27:53.396518 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-ee3b7a06-09b0-4e42-9446-286b3fb10fde tempest-VolumesSnapshotTestJSON-1750456804 tempest-VolumesSnapshotTestJSON-1750456804-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:27:53.397018 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-ee3b7a06-09b0-4e42-9446-286b3fb10fde tempest-VolumesSnapshotTestJSON-1750456804 tempest-VolumesSnapshotTestJSON-1750456804-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:27:53.397558 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-ee3b7a06-09b0-4e42-9446-286b3fb10fde tempest-VolumesSnapshotTestJSON-1750456804 tempest-VolumesSnapshotTestJSON-1750456804-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:27:53.398396 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-ee3b7a06-09b0-4e42-9446-286b3fb10fde tempest-VolumesSnapshotTestJSON-1750456804 tempest-VolumesSnapshotTestJSON-1750456804-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:27:53.398828 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-ee3b7a06-09b0-4e42-9446-286b3fb10fde tempest-VolumesSnapshotTestJSON-1750456804 tempest-VolumesSnapshotTestJSON-1750456804-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:27:53.399317 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-ee3b7a06-09b0-4e42-9446-286b3fb10fde tempest-VolumesSnapshotTestJSON-1750456804 tempest-VolumesSnapshotTestJSON-1750456804-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:27:53.541461 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-ee3b7a06-09b0-4e42-9446-286b3fb10fde tempest-VolumesSnapshotTestJSON-1750456804 tempest-VolumesSnapshotTestJSON-1750456804-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '75744eccc3d24d4c9282cbe3a3314489', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fa43d54225bf44b283f9db5ddbbb13ca', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:27:53.549456 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-ee3b7a06-09b0-4e42-9446-286b3fb10fde tempest-VolumesSnapshotTestJSON-1750456804 tempest-VolumesSnapshotTestJSON-1750456804-project] 10.222.0.52 "GET /compute/v2.1/servers/46dae88e-3429-47c7-9376-c4c0d9cddd29" status: 200 len: 1318 microversion: 2.1 time: 0.169098 Aug 27 02:27:53.550019 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 289/577] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:27:53 2022] GET /compute/v2.1/servers/46dae88e-3429-47c7-9376-c4c0d9cddd29 => generated 1318 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:27:54.447803 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [req-c7f99782-51c2-40ce-a719-b95f6c8ee402 req-35fe6038-2e12-4d0b-9397-4426d97ca51d service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "46dae88e-3429-47c7-9376-c4c0d9cddd29", "name": "network-vif-plugged", "status": "completed", "tag": "692449b2-a058-408c-bf04-f924a65e0f5a"}]} {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 27 02:27:54.452735 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG dbcounter [-] [120863] Writing DB stats nova_cell0:SELECT=2 {{(pid=120863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 27 02:27:54.453658 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG dbcounter [-] [120863] Writing DB stats nova_cell0:SELECT=6 {{(pid=120863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 27 02:27:54.461921 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [req-c7f99782-51c2-40ce-a719-b95f6c8ee402 req-35fe6038-2e12-4d0b-9397-4426d97ca51d service nova] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:27:54.462626 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [req-c7f99782-51c2-40ce-a719-b95f6c8ee402 req-35fe6038-2e12-4d0b-9397-4426d97ca51d service nova] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:27:54.462935 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [req-c7f99782-51c2-40ce-a719-b95f6c8ee402 req-35fe6038-2e12-4d0b-9397-4426d97ca51d service nova] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:27:54.474947 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.compute.server_external_events [req-c7f99782-51c2-40ce-a719-b95f6c8ee402 req-35fe6038-2e12-4d0b-9397-4426d97ca51d service nova] Creating event network-vif-plugged:692449b2-a058-408c-bf04-f924a65e0f5a for instance 46dae88e-3429-47c7-9376-c4c0d9cddd29 on hv-cinder-84630 Aug 27 02:27:54.480341 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [req-c7f99782-51c2-40ce-a719-b95f6c8ee402 req-35fe6038-2e12-4d0b-9397-4426d97ca51d service nova] 10.222.0.52 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.036228 Aug 27 02:27:54.480934 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 289/578] 10.222.0.52 () {64 vars in 1317 bytes} [Sat Aug 27 02:27:54 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 38 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Aug 27 02:27:54.572956 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-dd86e8aa-7310-475b-9596-079a59444adc tempest-VolumesSnapshotTestJSON-1750456804 tempest-VolumesSnapshotTestJSON-1750456804-project] Calling method '>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:27:54.573560 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-dd86e8aa-7310-475b-9596-079a59444adc tempest-VolumesSnapshotTestJSON-1750456804 tempest-VolumesSnapshotTestJSON-1750456804-project] [instance: 46dae88e-3429-47c7-9376-c4c0d9cddd29] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:27:54.580733 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-dd86e8aa-7310-475b-9596-079a59444adc tempest-VolumesSnapshotTestJSON-1750456804 tempest-VolumesSnapshotTestJSON-1750456804-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:27:54.581215 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-dd86e8aa-7310-475b-9596-079a59444adc tempest-VolumesSnapshotTestJSON-1750456804 tempest-VolumesSnapshotTestJSON-1750456804-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:27:54.581938 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-dd86e8aa-7310-475b-9596-079a59444adc tempest-VolumesSnapshotTestJSON-1750456804 tempest-VolumesSnapshotTestJSON-1750456804-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:27:54.582612 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-dd86e8aa-7310-475b-9596-079a59444adc tempest-VolumesSnapshotTestJSON-1750456804 tempest-VolumesSnapshotTestJSON-1750456804-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:27:54.587440 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-dd86e8aa-7310-475b-9596-079a59444adc tempest-VolumesSnapshotTestJSON-1750456804 tempest-VolumesSnapshotTestJSON-1750456804-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:27:54.587962 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-dd86e8aa-7310-475b-9596-079a59444adc tempest-VolumesSnapshotTestJSON-1750456804 tempest-VolumesSnapshotTestJSON-1750456804-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:27:54.749907 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-dd86e8aa-7310-475b-9596-079a59444adc tempest-VolumesSnapshotTestJSON-1750456804 tempest-VolumesSnapshotTestJSON-1750456804-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '75744eccc3d24d4c9282cbe3a3314489', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fa43d54225bf44b283f9db5ddbbb13ca', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:27:54.758023 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-dd86e8aa-7310-475b-9596-079a59444adc tempest-VolumesSnapshotTestJSON-1750456804 tempest-VolumesSnapshotTestJSON-1750456804-project] 10.222.0.52 "GET /compute/v2.1/servers/46dae88e-3429-47c7-9376-c4c0d9cddd29" status: 200 len: 1318 microversion: 2.1 time: 0.189359 Aug 27 02:27:54.758585 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 290/579] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:27:54 2022] GET /compute/v2.1/servers/46dae88e-3429-47c7-9376-c4c0d9cddd29 => generated 1318 bytes in 191 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:27:55.781335 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-463ede10-3883-4578-b203-98c7c81eb7bb tempest-VolumesSnapshotTestJSON-1750456804 tempest-VolumesSnapshotTestJSON-1750456804-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:27:55.782023 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-463ede10-3883-4578-b203-98c7c81eb7bb tempest-VolumesSnapshotTestJSON-1750456804 tempest-VolumesSnapshotTestJSON-1750456804-project] [instance: 46dae88e-3429-47c7-9376-c4c0d9cddd29] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:27:55.788699 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-463ede10-3883-4578-b203-98c7c81eb7bb tempest-VolumesSnapshotTestJSON-1750456804 tempest-VolumesSnapshotTestJSON-1750456804-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:27:55.789213 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-463ede10-3883-4578-b203-98c7c81eb7bb tempest-VolumesSnapshotTestJSON-1750456804 tempest-VolumesSnapshotTestJSON-1750456804-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:27:55.789649 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-463ede10-3883-4578-b203-98c7c81eb7bb tempest-VolumesSnapshotTestJSON-1750456804 tempest-VolumesSnapshotTestJSON-1750456804-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:27:55.790342 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-463ede10-3883-4578-b203-98c7c81eb7bb tempest-VolumesSnapshotTestJSON-1750456804 tempest-VolumesSnapshotTestJSON-1750456804-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:27:55.790792 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-463ede10-3883-4578-b203-98c7c81eb7bb tempest-VolumesSnapshotTestJSON-1750456804 tempest-VolumesSnapshotTestJSON-1750456804-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:27:55.791216 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-463ede10-3883-4578-b203-98c7c81eb7bb tempest-VolumesSnapshotTestJSON-1750456804 tempest-VolumesSnapshotTestJSON-1750456804-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:27:56.192160 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-463ede10-3883-4578-b203-98c7c81eb7bb tempest-VolumesSnapshotTestJSON-1750456804 tempest-VolumesSnapshotTestJSON-1750456804-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '75744eccc3d24d4c9282cbe3a3314489', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fa43d54225bf44b283f9db5ddbbb13ca', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:27:56.200856 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-463ede10-3883-4578-b203-98c7c81eb7bb tempest-VolumesSnapshotTestJSON-1750456804 tempest-VolumesSnapshotTestJSON-1750456804-project] 10.222.0.52 "GET /compute/v2.1/servers/46dae88e-3429-47c7-9376-c4c0d9cddd29" status: 200 len: 1318 microversion: 2.1 time: 0.424385 Aug 27 02:27:56.201469 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 290/580] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:27:55 2022] GET /compute/v2.1/servers/46dae88e-3429-47c7-9376-c4c0d9cddd29 => generated 1318 bytes in 426 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:27:57.225939 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-64a7a951-61d7-42e1-b2c2-4fccf3e9ae61 tempest-VolumesSnapshotTestJSON-1750456804 tempest-VolumesSnapshotTestJSON-1750456804-project] Calling method '>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:27:57.226954 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-64a7a951-61d7-42e1-b2c2-4fccf3e9ae61 tempest-VolumesSnapshotTestJSON-1750456804 tempest-VolumesSnapshotTestJSON-1750456804-project] [instance: 46dae88e-3429-47c7-9376-c4c0d9cddd29] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:27:57.233872 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-64a7a951-61d7-42e1-b2c2-4fccf3e9ae61 tempest-VolumesSnapshotTestJSON-1750456804 tempest-VolumesSnapshotTestJSON-1750456804-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:27:57.234459 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-64a7a951-61d7-42e1-b2c2-4fccf3e9ae61 tempest-VolumesSnapshotTestJSON-1750456804 tempest-VolumesSnapshotTestJSON-1750456804-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:27:57.234903 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-64a7a951-61d7-42e1-b2c2-4fccf3e9ae61 tempest-VolumesSnapshotTestJSON-1750456804 tempest-VolumesSnapshotTestJSON-1750456804-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:27:57.235597 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-64a7a951-61d7-42e1-b2c2-4fccf3e9ae61 tempest-VolumesSnapshotTestJSON-1750456804 tempest-VolumesSnapshotTestJSON-1750456804-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:27:57.236054 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-64a7a951-61d7-42e1-b2c2-4fccf3e9ae61 tempest-VolumesSnapshotTestJSON-1750456804 tempest-VolumesSnapshotTestJSON-1750456804-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:27:57.236469 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-64a7a951-61d7-42e1-b2c2-4fccf3e9ae61 tempest-VolumesSnapshotTestJSON-1750456804 tempest-VolumesSnapshotTestJSON-1750456804-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:27:57.385448 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-64a7a951-61d7-42e1-b2c2-4fccf3e9ae61 tempest-VolumesSnapshotTestJSON-1750456804 tempest-VolumesSnapshotTestJSON-1750456804-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '75744eccc3d24d4c9282cbe3a3314489', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fa43d54225bf44b283f9db5ddbbb13ca', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:27:57.394692 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-64a7a951-61d7-42e1-b2c2-4fccf3e9ae61 tempest-VolumesSnapshotTestJSON-1750456804 tempest-VolumesSnapshotTestJSON-1750456804-project] 10.222.0.52 "GET /compute/v2.1/servers/46dae88e-3429-47c7-9376-c4c0d9cddd29" status: 200 len: 1507 microversion: 2.1 time: 0.175188 Aug 27 02:27:57.395453 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 291/581] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:27:57 2022] GET /compute/v2.1/servers/46dae88e-3429-47c7-9376-c4c0d9cddd29 => generated 1507 bytes in 177 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:27:57.417541 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-bc49d822-3ec8-42df-b263-99355ed08f44 tempest-VolumesSnapshotTestJSON-1750456804 tempest-VolumesSnapshotTestJSON-1750456804-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "35da299d-3a4d-4a2b-a065-2c4fcbbdef52", "device": "/dev/sdb"}} {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 27 02:27:57.421088 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-bc49d822-3ec8-42df-b263-99355ed08f44 tempest-VolumesSnapshotTestJSON-1750456804 tempest-VolumesSnapshotTestJSON-1750456804-project] [instance: 46dae88e-3429-47c7-9376-c4c0d9cddd29] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:27:57.429555 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-bc49d822-3ec8-42df-b263-99355ed08f44 tempest-VolumesSnapshotTestJSON-1750456804 tempest-VolumesSnapshotTestJSON-1750456804-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:27:57.430109 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-bc49d822-3ec8-42df-b263-99355ed08f44 tempest-VolumesSnapshotTestJSON-1750456804 tempest-VolumesSnapshotTestJSON-1750456804-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:27:57.430595 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-bc49d822-3ec8-42df-b263-99355ed08f44 tempest-VolumesSnapshotTestJSON-1750456804 tempest-VolumesSnapshotTestJSON-1750456804-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:27:57.431599 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-bc49d822-3ec8-42df-b263-99355ed08f44 tempest-VolumesSnapshotTestJSON-1750456804 tempest-VolumesSnapshotTestJSON-1750456804-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:27:57.432169 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-bc49d822-3ec8-42df-b263-99355ed08f44 tempest-VolumesSnapshotTestJSON-1750456804 tempest-VolumesSnapshotTestJSON-1750456804-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:27:57.432688 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-bc49d822-3ec8-42df-b263-99355ed08f44 tempest-VolumesSnapshotTestJSON-1750456804 tempest-VolumesSnapshotTestJSON-1750456804-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:27:57.843377 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-bc49d822-3ec8-42df-b263-99355ed08f44 tempest-VolumesSnapshotTestJSON-1750456804 tempest-VolumesSnapshotTestJSON-1750456804-project] 10.222.0.52 "POST /compute/v2.1/servers/46dae88e-3429-47c7-9376-c4c0d9cddd29/os-volume_attachments" status: 200 len: 194 microversion: 2.1 time: 0.430660 Aug 27 02:27:57.843983 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 291/582] 10.222.0.52 () {60 vars in 1323 bytes} [Sat Aug 27 02:27:57 2022] POST /compute/v2.1/servers/46dae88e-3429-47c7-9376-c4c0d9cddd29/os-volume_attachments => generated 194 bytes in 433 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 27 02:28:04.009754 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-9e3827dc-9070-4e87-b75a-0ce7db72ebb2 tempest-VolumesNegativeTest-1839426864 tempest-VolumesNegativeTest-1839426864-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-VolumesNegativeTest-instance-118619479", "imageRef": "9fc30b36-b827-46f9-892d-30e808bc2b8a", "flavorRef": "452"}} {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 27 02:28:04.075936 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.network.neutron [None req-9e3827dc-9070-4e87-b75a-0ce7db72ebb2 tempest-VolumesNegativeTest-1839426864 tempest-VolumesNegativeTest-1839426864-project] validate_networks() for None {{(pid=120864) validate_networks /opt/stack/nova/nova/network/neutron.py:2626}} Aug 27 02:28:04.384363 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.virt.hardware [None req-9e3827dc-9070-4e87-b75a-0ce7db72ebb2 tempest-VolumesNegativeTest-1839426864 tempest-VolumesNegativeTest-1839426864-project] Flavor limits 0:0:0 {{(pid=120864) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Aug 27 02:28:04.384782 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.virt.hardware [None req-9e3827dc-9070-4e87-b75a-0ce7db72ebb2 tempest-VolumesNegativeTest-1839426864 tempest-VolumesNegativeTest-1839426864-project] Image limits 0:0:0 {{(pid=120864) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Aug 27 02:28:04.385313 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.virt.hardware [None req-9e3827dc-9070-4e87-b75a-0ce7db72ebb2 tempest-VolumesNegativeTest-1839426864 tempest-VolumesNegativeTest-1839426864-project] Flavor pref 0:0:0 {{(pid=120864) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Aug 27 02:28:04.385720 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.virt.hardware [None req-9e3827dc-9070-4e87-b75a-0ce7db72ebb2 tempest-VolumesNegativeTest-1839426864 tempest-VolumesNegativeTest-1839426864-project] Image pref 0:0:0 {{(pid=120864) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Aug 27 02:28:04.386154 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.virt.hardware [None req-9e3827dc-9070-4e87-b75a-0ce7db72ebb2 tempest-VolumesNegativeTest-1839426864 tempest-VolumesNegativeTest-1839426864-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=120864) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Aug 27 02:28:04.395669 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-9e3827dc-9070-4e87-b75a-0ce7db72ebb2 tempest-VolumesNegativeTest-1839426864 tempest-VolumesNegativeTest-1839426864-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:28:04.396446 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-9e3827dc-9070-4e87-b75a-0ce7db72ebb2 tempest-VolumesNegativeTest-1839426864 tempest-VolumesNegativeTest-1839426864-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:28:04.396856 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-9e3827dc-9070-4e87-b75a-0ce7db72ebb2 tempest-VolumesNegativeTest-1839426864 tempest-VolumesNegativeTest-1839426864-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:28:04.397644 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-9e3827dc-9070-4e87-b75a-0ce7db72ebb2 tempest-VolumesNegativeTest-1839426864 tempest-VolumesNegativeTest-1839426864-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:28:04.398097 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-9e3827dc-9070-4e87-b75a-0ce7db72ebb2 tempest-VolumesNegativeTest-1839426864 tempest-VolumesNegativeTest-1839426864-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:28:04.398523 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-9e3827dc-9070-4e87-b75a-0ce7db72ebb2 tempest-VolumesNegativeTest-1839426864 tempest-VolumesNegativeTest-1839426864-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:28:04.415975 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.quota [None req-9e3827dc-9070-4e87-b75a-0ce7db72ebb2 tempest-VolumesNegativeTest-1839426864 tempest-VolumesNegativeTest-1839426864-project] Getting quotas for project e1d1bead2e4f4102a53f582a4815661d. Resources: {'cores', 'ram', 'instances'} {{(pid=120864) _get_quotas /opt/stack/nova/nova/quota.py:393}} Aug 27 02:28:04.421047 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.quota [None req-9e3827dc-9070-4e87-b75a-0ce7db72ebb2 tempest-VolumesNegativeTest-1839426864 tempest-VolumesNegativeTest-1839426864-project] Getting quotas for user 544e6c056b854bfdb9e117be686635a0 and project e1d1bead2e4f4102a53f582a4815661d. Resources: {'cores', 'ram', 'instances'} {{(pid=120864) _get_quotas /opt/stack/nova/nova/quota.py:383}} Aug 27 02:28:04.433930 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-9e3827dc-9070-4e87-b75a-0ce7db72ebb2 tempest-VolumesNegativeTest-1839426864 tempest-VolumesNegativeTest-1839426864-project] Going to run 1 instances... {{(pid=120864) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Aug 27 02:28:04.436404 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-9e3827dc-9070-4e87-b75a-0ce7db72ebb2 tempest-VolumesNegativeTest-1839426864 tempest-VolumesNegativeTest-1839426864-project] [instance: ac07d059-1555-4f61-9a85-28396691a934] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='9fc30b36-b827-46f9-892d-30e808bc2b8a',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=120864) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Aug 27 02:28:04.486522 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-9e3827dc-9070-4e87-b75a-0ce7db72ebb2 tempest-VolumesNegativeTest-1839426864 tempest-VolumesNegativeTest-1839426864-project] 10.222.0.52 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 0.480823 Aug 27 02:28:04.487613 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 292/583] 10.222.0.52 () {60 vars in 1147 bytes} [Sat Aug 27 02:28:04 2022] POST /compute/v2.1/servers => generated 380 bytes in 483 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Aug 27 02:28:04.505057 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-ac2d0697-ad23-4ac2-9a33-7104f619499a tempest-VolumesNegativeTest-1839426864 tempest-VolumesNegativeTest-1839426864-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:28:04.506027 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-ac2d0697-ad23-4ac2-9a33-7104f619499a tempest-VolumesNegativeTest-1839426864 tempest-VolumesNegativeTest-1839426864-project] [instance: ac07d059-1555-4f61-9a85-28396691a934] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:28:04.567579 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-ac2d0697-ad23-4ac2-9a33-7104f619499a tempest-VolumesNegativeTest-1839426864 tempest-VolumesNegativeTest-1839426864-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '544e6c056b854bfdb9e117be686635a0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d1bead2e4f4102a53f582a4815661d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:28:04.575332 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-ac2d0697-ad23-4ac2-9a33-7104f619499a tempest-VolumesNegativeTest-1839426864 tempest-VolumesNegativeTest-1839426864-project] 10.222.0.52 "GET /compute/v2.1/servers/ac07d059-1555-4f61-9a85-28396691a934" status: 200 len: 1214 microversion: 2.1 time: 0.078349 Aug 27 02:28:04.589145 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 292/584] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:28:04 2022] GET /compute/v2.1/servers/ac07d059-1555-4f61-9a85-28396691a934 => generated 1214 bytes in 80 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 27 02:28:05.605978 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-6221aae8-5b73-40a9-a7c9-19d075dd5fe3 tempest-VolumesNegativeTest-1839426864 tempest-VolumesNegativeTest-1839426864-project] Calling method '>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:28:05.606646 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-6221aae8-5b73-40a9-a7c9-19d075dd5fe3 tempest-VolumesNegativeTest-1839426864 tempest-VolumesNegativeTest-1839426864-project] [instance: ac07d059-1555-4f61-9a85-28396691a934] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:28:05.613835 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-6221aae8-5b73-40a9-a7c9-19d075dd5fe3 tempest-VolumesNegativeTest-1839426864 tempest-VolumesNegativeTest-1839426864-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:28:05.614330 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-6221aae8-5b73-40a9-a7c9-19d075dd5fe3 tempest-VolumesNegativeTest-1839426864 tempest-VolumesNegativeTest-1839426864-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:28:05.615240 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-6221aae8-5b73-40a9-a7c9-19d075dd5fe3 tempest-VolumesNegativeTest-1839426864 tempest-VolumesNegativeTest-1839426864-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:28:05.615921 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-6221aae8-5b73-40a9-a7c9-19d075dd5fe3 tempest-VolumesNegativeTest-1839426864 tempest-VolumesNegativeTest-1839426864-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:28:05.616370 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-6221aae8-5b73-40a9-a7c9-19d075dd5fe3 tempest-VolumesNegativeTest-1839426864 tempest-VolumesNegativeTest-1839426864-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:28:05.616783 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-6221aae8-5b73-40a9-a7c9-19d075dd5fe3 tempest-VolumesNegativeTest-1839426864 tempest-VolumesNegativeTest-1839426864-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:28:05.696385 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-6221aae8-5b73-40a9-a7c9-19d075dd5fe3 tempest-VolumesNegativeTest-1839426864 tempest-VolumesNegativeTest-1839426864-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '544e6c056b854bfdb9e117be686635a0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d1bead2e4f4102a53f582a4815661d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:28:05.704637 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-6221aae8-5b73-40a9-a7c9-19d075dd5fe3 tempest-VolumesNegativeTest-1839426864 tempest-VolumesNegativeTest-1839426864-project] 10.222.0.52 "GET /compute/v2.1/servers/ac07d059-1555-4f61-9a85-28396691a934" status: 200 len: 1266 microversion: 2.1 time: 0.104199 Aug 27 02:28:05.705229 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 293/585] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:28:05 2022] GET /compute/v2.1/servers/ac07d059-1555-4f61-9a85-28396691a934 => generated 1266 bytes in 106 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:28:06.727949 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-8d9fcc82-ab4e-45a7-98be-5dc94c37bee8 tempest-VolumesNegativeTest-1839426864 tempest-VolumesNegativeTest-1839426864-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:28:06.728549 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-8d9fcc82-ab4e-45a7-98be-5dc94c37bee8 tempest-VolumesNegativeTest-1839426864 tempest-VolumesNegativeTest-1839426864-project] [instance: ac07d059-1555-4f61-9a85-28396691a934] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:28:06.736164 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-8d9fcc82-ab4e-45a7-98be-5dc94c37bee8 tempest-VolumesNegativeTest-1839426864 tempest-VolumesNegativeTest-1839426864-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:28:06.736645 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-8d9fcc82-ab4e-45a7-98be-5dc94c37bee8 tempest-VolumesNegativeTest-1839426864 tempest-VolumesNegativeTest-1839426864-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:28:06.737085 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-8d9fcc82-ab4e-45a7-98be-5dc94c37bee8 tempest-VolumesNegativeTest-1839426864 tempest-VolumesNegativeTest-1839426864-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:28:06.737802 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-8d9fcc82-ab4e-45a7-98be-5dc94c37bee8 tempest-VolumesNegativeTest-1839426864 tempest-VolumesNegativeTest-1839426864-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:28:06.738259 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-8d9fcc82-ab4e-45a7-98be-5dc94c37bee8 tempest-VolumesNegativeTest-1839426864 tempest-VolumesNegativeTest-1839426864-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:28:06.738684 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-8d9fcc82-ab4e-45a7-98be-5dc94c37bee8 tempest-VolumesNegativeTest-1839426864 tempest-VolumesNegativeTest-1839426864-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:28:06.803364 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-8d9fcc82-ab4e-45a7-98be-5dc94c37bee8 tempest-VolumesNegativeTest-1839426864 tempest-VolumesNegativeTest-1839426864-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '544e6c056b854bfdb9e117be686635a0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d1bead2e4f4102a53f582a4815661d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:28:06.811630 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-8d9fcc82-ab4e-45a7-98be-5dc94c37bee8 tempest-VolumesNegativeTest-1839426864 tempest-VolumesNegativeTest-1839426864-project] 10.222.0.52 "GET /compute/v2.1/servers/ac07d059-1555-4f61-9a85-28396691a934" status: 200 len: 1274 microversion: 2.1 time: 0.088225 Aug 27 02:28:06.812232 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 293/586] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:28:06 2022] GET /compute/v2.1/servers/ac07d059-1555-4f61-9a85-28396691a934 => generated 1274 bytes in 89 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:28:07.834591 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-8245cf93-81cd-4cf1-8b19-ac4d2a51ed2e tempest-VolumesNegativeTest-1839426864 tempest-VolumesNegativeTest-1839426864-project] Calling method '>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:28:07.835466 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-8245cf93-81cd-4cf1-8b19-ac4d2a51ed2e tempest-VolumesNegativeTest-1839426864 tempest-VolumesNegativeTest-1839426864-project] [instance: ac07d059-1555-4f61-9a85-28396691a934] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:28:07.841901 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-8245cf93-81cd-4cf1-8b19-ac4d2a51ed2e tempest-VolumesNegativeTest-1839426864 tempest-VolumesNegativeTest-1839426864-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:28:07.842364 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-8245cf93-81cd-4cf1-8b19-ac4d2a51ed2e tempest-VolumesNegativeTest-1839426864 tempest-VolumesNegativeTest-1839426864-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:28:07.842794 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-8245cf93-81cd-4cf1-8b19-ac4d2a51ed2e tempest-VolumesNegativeTest-1839426864 tempest-VolumesNegativeTest-1839426864-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:28:07.843483 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-8245cf93-81cd-4cf1-8b19-ac4d2a51ed2e tempest-VolumesNegativeTest-1839426864 tempest-VolumesNegativeTest-1839426864-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:28:07.843925 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-8245cf93-81cd-4cf1-8b19-ac4d2a51ed2e tempest-VolumesNegativeTest-1839426864 tempest-VolumesNegativeTest-1839426864-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:28:07.844336 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-8245cf93-81cd-4cf1-8b19-ac4d2a51ed2e tempest-VolumesNegativeTest-1839426864 tempest-VolumesNegativeTest-1839426864-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:28:07.907177 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-8245cf93-81cd-4cf1-8b19-ac4d2a51ed2e tempest-VolumesNegativeTest-1839426864 tempest-VolumesNegativeTest-1839426864-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '544e6c056b854bfdb9e117be686635a0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d1bead2e4f4102a53f582a4815661d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:28:07.915548 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-8245cf93-81cd-4cf1-8b19-ac4d2a51ed2e tempest-VolumesNegativeTest-1839426864 tempest-VolumesNegativeTest-1839426864-project] 10.222.0.52 "GET /compute/v2.1/servers/ac07d059-1555-4f61-9a85-28396691a934" status: 200 len: 1274 microversion: 2.1 time: 0.085328 Aug 27 02:28:07.916137 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 294/587] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:28:07 2022] GET /compute/v2.1/servers/ac07d059-1555-4f61-9a85-28396691a934 => generated 1274 bytes in 87 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:28:08.937193 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-5997444d-69d1-4641-9ee3-7a336e8083e7 tempest-VolumesNegativeTest-1839426864 tempest-VolumesNegativeTest-1839426864-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:28:08.937757 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-5997444d-69d1-4641-9ee3-7a336e8083e7 tempest-VolumesNegativeTest-1839426864 tempest-VolumesNegativeTest-1839426864-project] [instance: ac07d059-1555-4f61-9a85-28396691a934] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:28:08.944998 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-5997444d-69d1-4641-9ee3-7a336e8083e7 tempest-VolumesNegativeTest-1839426864 tempest-VolumesNegativeTest-1839426864-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:28:08.945512 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-5997444d-69d1-4641-9ee3-7a336e8083e7 tempest-VolumesNegativeTest-1839426864 tempest-VolumesNegativeTest-1839426864-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:28:08.945946 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-5997444d-69d1-4641-9ee3-7a336e8083e7 tempest-VolumesNegativeTest-1839426864 tempest-VolumesNegativeTest-1839426864-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:28:08.946639 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-5997444d-69d1-4641-9ee3-7a336e8083e7 tempest-VolumesNegativeTest-1839426864 tempest-VolumesNegativeTest-1839426864-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:28:08.947092 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-5997444d-69d1-4641-9ee3-7a336e8083e7 tempest-VolumesNegativeTest-1839426864 tempest-VolumesNegativeTest-1839426864-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:28:08.947519 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-5997444d-69d1-4641-9ee3-7a336e8083e7 tempest-VolumesNegativeTest-1839426864 tempest-VolumesNegativeTest-1839426864-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:28:09.011376 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-5997444d-69d1-4641-9ee3-7a336e8083e7 tempest-VolumesNegativeTest-1839426864 tempest-VolumesNegativeTest-1839426864-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '544e6c056b854bfdb9e117be686635a0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d1bead2e4f4102a53f582a4815661d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:28:09.019735 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-5997444d-69d1-4641-9ee3-7a336e8083e7 tempest-VolumesNegativeTest-1839426864 tempest-VolumesNegativeTest-1839426864-project] 10.222.0.52 "GET /compute/v2.1/servers/ac07d059-1555-4f61-9a85-28396691a934" status: 200 len: 1274 microversion: 2.1 time: 0.086631 Aug 27 02:28:09.020372 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 294/588] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:28:08 2022] GET /compute/v2.1/servers/ac07d059-1555-4f61-9a85-28396691a934 => generated 1274 bytes in 88 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:28:10.041393 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-401a643c-a664-4e48-a694-c05bd4b893e0 tempest-VolumesNegativeTest-1839426864 tempest-VolumesNegativeTest-1839426864-project] Calling method '>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:28:10.041977 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-401a643c-a664-4e48-a694-c05bd4b893e0 tempest-VolumesNegativeTest-1839426864 tempest-VolumesNegativeTest-1839426864-project] [instance: ac07d059-1555-4f61-9a85-28396691a934] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:28:10.048777 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-401a643c-a664-4e48-a694-c05bd4b893e0 tempest-VolumesNegativeTest-1839426864 tempest-VolumesNegativeTest-1839426864-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:28:10.049282 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-401a643c-a664-4e48-a694-c05bd4b893e0 tempest-VolumesNegativeTest-1839426864 tempest-VolumesNegativeTest-1839426864-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:28:10.049717 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-401a643c-a664-4e48-a694-c05bd4b893e0 tempest-VolumesNegativeTest-1839426864 tempest-VolumesNegativeTest-1839426864-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:28:10.050406 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-401a643c-a664-4e48-a694-c05bd4b893e0 tempest-VolumesNegativeTest-1839426864 tempest-VolumesNegativeTest-1839426864-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:28:10.050857 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-401a643c-a664-4e48-a694-c05bd4b893e0 tempest-VolumesNegativeTest-1839426864 tempest-VolumesNegativeTest-1839426864-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:28:10.051284 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-401a643c-a664-4e48-a694-c05bd4b893e0 tempest-VolumesNegativeTest-1839426864 tempest-VolumesNegativeTest-1839426864-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:28:10.115314 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-401a643c-a664-4e48-a694-c05bd4b893e0 tempest-VolumesNegativeTest-1839426864 tempest-VolumesNegativeTest-1839426864-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '544e6c056b854bfdb9e117be686635a0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d1bead2e4f4102a53f582a4815661d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:28:10.123489 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-401a643c-a664-4e48-a694-c05bd4b893e0 tempest-VolumesNegativeTest-1839426864 tempest-VolumesNegativeTest-1839426864-project] 10.222.0.52 "GET /compute/v2.1/servers/ac07d059-1555-4f61-9a85-28396691a934" status: 200 len: 1272 microversion: 2.1 time: 0.086063 Aug 27 02:28:10.124062 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 295/589] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:28:10 2022] GET /compute/v2.1/servers/ac07d059-1555-4f61-9a85-28396691a934 => generated 1272 bytes in 87 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:28:11.012035 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-28c28366-09e9-4fb6-a2c3-d7aa49dcaecb tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:28:11.012621 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-28c28366-09e9-4fb6-a2c3-d7aa49dcaecb tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] [instance: 56dbfa6a-5b6f-4b82-977b-af735e6a6ac9] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:28:11.023780 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-28c28366-09e9-4fb6-a2c3-d7aa49dcaecb tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:28:11.025101 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-28c28366-09e9-4fb6-a2c3-d7aa49dcaecb tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:28:11.025682 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-28c28366-09e9-4fb6-a2c3-d7aa49dcaecb tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:28:11.026818 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-28c28366-09e9-4fb6-a2c3-d7aa49dcaecb tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:28:11.027092 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-28c28366-09e9-4fb6-a2c3-d7aa49dcaecb tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:28:11.027583 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-28c28366-09e9-4fb6-a2c3-d7aa49dcaecb tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:28:11.052701 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-28c28366-09e9-4fb6-a2c3-d7aa49dcaecb tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] 10.222.0.52 "GET /compute/v2.1/servers/56dbfa6a-5b6f-4b82-977b-af735e6a6ac9/ips" status: 200 len: 152 microversion: 2.20 time: 0.044486 Aug 27 02:28:11.057331 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 295/590] 10.222.0.52 () {60 vars in 1289 bytes} [Sat Aug 27 02:28:11 2022] GET /compute/v2.1/servers/56dbfa6a-5b6f-4b82-977b-af735e6a6ac9/ips => generated 152 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Aug 27 02:28:11.145034 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-4159ff7a-6529-473a-bab4-76953f7b16af tempest-VolumesNegativeTest-1839426864 tempest-VolumesNegativeTest-1839426864-project] Calling method '>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:28:11.146268 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-4159ff7a-6529-473a-bab4-76953f7b16af tempest-VolumesNegativeTest-1839426864 tempest-VolumesNegativeTest-1839426864-project] [instance: ac07d059-1555-4f61-9a85-28396691a934] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:28:11.152951 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-4159ff7a-6529-473a-bab4-76953f7b16af tempest-VolumesNegativeTest-1839426864 tempest-VolumesNegativeTest-1839426864-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:28:11.153471 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-4159ff7a-6529-473a-bab4-76953f7b16af tempest-VolumesNegativeTest-1839426864 tempest-VolumesNegativeTest-1839426864-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:28:11.153897 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-4159ff7a-6529-473a-bab4-76953f7b16af tempest-VolumesNegativeTest-1839426864 tempest-VolumesNegativeTest-1839426864-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:28:11.154589 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-4159ff7a-6529-473a-bab4-76953f7b16af tempest-VolumesNegativeTest-1839426864 tempest-VolumesNegativeTest-1839426864-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:28:11.155032 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-4159ff7a-6529-473a-bab4-76953f7b16af tempest-VolumesNegativeTest-1839426864 tempest-VolumesNegativeTest-1839426864-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:28:11.155437 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-4159ff7a-6529-473a-bab4-76953f7b16af tempest-VolumesNegativeTest-1839426864 tempest-VolumesNegativeTest-1839426864-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:28:11.223274 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-4159ff7a-6529-473a-bab4-76953f7b16af tempest-VolumesNegativeTest-1839426864 tempest-VolumesNegativeTest-1839426864-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '544e6c056b854bfdb9e117be686635a0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d1bead2e4f4102a53f582a4815661d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:28:11.232045 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-4159ff7a-6529-473a-bab4-76953f7b16af tempest-VolumesNegativeTest-1839426864 tempest-VolumesNegativeTest-1839426864-project] 10.222.0.52 "GET /compute/v2.1/servers/ac07d059-1555-4f61-9a85-28396691a934" status: 200 len: 1272 microversion: 2.1 time: 0.090678 Aug 27 02:28:11.232882 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 296/591] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:28:11 2022] GET /compute/v2.1/servers/ac07d059-1555-4f61-9a85-28396691a934 => generated 1272 bytes in 92 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:28:12.253926 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-eaaa885a-6001-42ef-98db-fbb309c3c2c0 tempest-VolumesNegativeTest-1839426864 tempest-VolumesNegativeTest-1839426864-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:28:12.254868 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-eaaa885a-6001-42ef-98db-fbb309c3c2c0 tempest-VolumesNegativeTest-1839426864 tempest-VolumesNegativeTest-1839426864-project] [instance: ac07d059-1555-4f61-9a85-28396691a934] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:28:12.257893 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG dbcounter [-] [120863] Writing DB stats nova_api:SELECT=40,nova_api:INSERT=3 {{(pid=120863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 27 02:28:12.264927 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-eaaa885a-6001-42ef-98db-fbb309c3c2c0 tempest-VolumesNegativeTest-1839426864 tempest-VolumesNegativeTest-1839426864-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:28:12.265457 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-eaaa885a-6001-42ef-98db-fbb309c3c2c0 tempest-VolumesNegativeTest-1839426864 tempest-VolumesNegativeTest-1839426864-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:28:12.265926 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-eaaa885a-6001-42ef-98db-fbb309c3c2c0 tempest-VolumesNegativeTest-1839426864 tempest-VolumesNegativeTest-1839426864-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:28:12.266724 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-eaaa885a-6001-42ef-98db-fbb309c3c2c0 tempest-VolumesNegativeTest-1839426864 tempest-VolumesNegativeTest-1839426864-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:28:12.267211 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-eaaa885a-6001-42ef-98db-fbb309c3c2c0 tempest-VolumesNegativeTest-1839426864 tempest-VolumesNegativeTest-1839426864-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:28:12.267638 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-eaaa885a-6001-42ef-98db-fbb309c3c2c0 tempest-VolumesNegativeTest-1839426864 tempest-VolumesNegativeTest-1839426864-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:28:12.271262 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG dbcounter [-] [120863] Writing DB stats nova_cell1:SELECT=64,nova_cell1:UPDATE=2,nova_cell1:INSERT=1 {{(pid=120863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 27 02:28:12.430716 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-eaaa885a-6001-42ef-98db-fbb309c3c2c0 tempest-VolumesNegativeTest-1839426864 tempest-VolumesNegativeTest-1839426864-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '544e6c056b854bfdb9e117be686635a0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d1bead2e4f4102a53f582a4815661d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:28:12.439934 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-eaaa885a-6001-42ef-98db-fbb309c3c2c0 tempest-VolumesNegativeTest-1839426864 tempest-VolumesNegativeTest-1839426864-project] 10.222.0.52 "GET /compute/v2.1/servers/ac07d059-1555-4f61-9a85-28396691a934" status: 200 len: 1314 microversion: 2.1 time: 0.190259 Aug 27 02:28:12.440676 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 296/592] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:28:12 2022] GET /compute/v2.1/servers/ac07d059-1555-4f61-9a85-28396691a934 => generated 1314 bytes in 192 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 27 02:28:12.465915 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [req-f766e988-15b0-40e4-a2bf-140549df2c04 req-851e2936-c522-4f6b-9965-34268a30a704 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "ac07d059-1555-4f61-9a85-28396691a934", "tag": "42163688-0f26-433a-b60b-97256fc0be15"}]} {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 27 02:28:12.476185 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [req-f766e988-15b0-40e4-a2bf-140549df2c04 req-851e2936-c522-4f6b-9965-34268a30a704 service nova] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:28:12.476666 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [req-f766e988-15b0-40e4-a2bf-140549df2c04 req-851e2936-c522-4f6b-9965-34268a30a704 service nova] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:28:12.477093 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [req-f766e988-15b0-40e4-a2bf-140549df2c04 req-851e2936-c522-4f6b-9965-34268a30a704 service nova] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:28:12.487921 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.compute.server_external_events [req-f766e988-15b0-40e4-a2bf-140549df2c04 req-851e2936-c522-4f6b-9965-34268a30a704 service nova] Creating event network-changed:42163688-0f26-433a-b60b-97256fc0be15 for instance ac07d059-1555-4f61-9a85-28396691a934 on hv-cinder-84630 Aug 27 02:28:12.493903 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [req-f766e988-15b0-40e4-a2bf-140549df2c04 req-851e2936-c522-4f6b-9965-34268a30a704 service nova] 10.222.0.52 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.182201 Aug 27 02:28:12.494496 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 297/593] 10.222.0.52 () {64 vars in 1317 bytes} [Sat Aug 27 02:28:12 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 184 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 27 02:28:13.462569 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-18a735c5-e4e0-47a1-9e71-5f327245d34f tempest-VolumesNegativeTest-1839426864 tempest-VolumesNegativeTest-1839426864-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:28:13.463184 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-18a735c5-e4e0-47a1-9e71-5f327245d34f tempest-VolumesNegativeTest-1839426864 tempest-VolumesNegativeTest-1839426864-project] [instance: ac07d059-1555-4f61-9a85-28396691a934] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:28:13.469986 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-18a735c5-e4e0-47a1-9e71-5f327245d34f tempest-VolumesNegativeTest-1839426864 tempest-VolumesNegativeTest-1839426864-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:28:13.470484 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-18a735c5-e4e0-47a1-9e71-5f327245d34f tempest-VolumesNegativeTest-1839426864 tempest-VolumesNegativeTest-1839426864-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:28:13.470939 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-18a735c5-e4e0-47a1-9e71-5f327245d34f tempest-VolumesNegativeTest-1839426864 tempest-VolumesNegativeTest-1839426864-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:28:13.471638 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-18a735c5-e4e0-47a1-9e71-5f327245d34f tempest-VolumesNegativeTest-1839426864 tempest-VolumesNegativeTest-1839426864-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:28:13.472234 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-18a735c5-e4e0-47a1-9e71-5f327245d34f tempest-VolumesNegativeTest-1839426864 tempest-VolumesNegativeTest-1839426864-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:28:13.472519 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-18a735c5-e4e0-47a1-9e71-5f327245d34f tempest-VolumesNegativeTest-1839426864 tempest-VolumesNegativeTest-1839426864-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:28:13.605698 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-18a735c5-e4e0-47a1-9e71-5f327245d34f tempest-VolumesNegativeTest-1839426864 tempest-VolumesNegativeTest-1839426864-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '544e6c056b854bfdb9e117be686635a0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d1bead2e4f4102a53f582a4815661d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:28:13.613787 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-18a735c5-e4e0-47a1-9e71-5f327245d34f tempest-VolumesNegativeTest-1839426864 tempest-VolumesNegativeTest-1839426864-project] 10.222.0.52 "GET /compute/v2.1/servers/ac07d059-1555-4f61-9a85-28396691a934" status: 200 len: 1314 microversion: 2.1 time: 0.155742 Aug 27 02:28:13.614341 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 297/594] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:28:13 2022] GET /compute/v2.1/servers/ac07d059-1555-4f61-9a85-28396691a934 => generated 1314 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:28:14.084691 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-7753880c-dc0d-42b7-a96e-a0415c93ede4 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Action: 'action', calling method: >, body: {"shelve": {}} {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 27 02:28:14.085322 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-7753880c-dc0d-42b7-a96e-a0415c93ede4 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] [instance: 56dbfa6a-5b6f-4b82-977b-af735e6a6ac9] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:28:14.088009 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG dbcounter [-] [120864] Writing DB stats nova_api:SELECT=42,nova_api:INSERT=3 {{(pid=120864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 27 02:28:14.093999 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-7753880c-dc0d-42b7-a96e-a0415c93ede4 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:28:14.094484 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-7753880c-dc0d-42b7-a96e-a0415c93ede4 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:28:14.094929 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-7753880c-dc0d-42b7-a96e-a0415c93ede4 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:28:14.095626 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-7753880c-dc0d-42b7-a96e-a0415c93ede4 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:28:14.096073 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-7753880c-dc0d-42b7-a96e-a0415c93ede4 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:28:14.096494 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-7753880c-dc0d-42b7-a96e-a0415c93ede4 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:28:14.100199 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG dbcounter [-] [120864] Writing DB stats nova_cell1:SELECT=71 {{(pid=120864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 27 02:28:14.122642 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.objects.instance [None req-7753880c-dc0d-42b7-a96e-a0415c93ede4 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lazy-loading 'flavor' on Instance uuid 56dbfa6a-5b6f-4b82-977b-af735e6a6ac9 {{(pid=120864) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Aug 27 02:28:14.374391 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-7753880c-dc0d-42b7-a96e-a0415c93ede4 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] 10.222.0.52 "POST /compute/v2.1/servers/56dbfa6a-5b6f-4b82-977b-af735e6a6ac9/action" status: 202 len: 0 microversion: 2.20 time: 0.294244 Aug 27 02:28:14.374981 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 298/595] 10.222.0.52 () {62 vars in 1319 bytes} [Sat Aug 27 02:28:14 2022] POST /compute/v2.1/servers/56dbfa6a-5b6f-4b82-977b-af735e6a6ac9/action => generated 0 bytes in 296 msecs (HTTP/1.1 202) 9 headers in 363 bytes (1 switches on core 0) Aug 27 02:28:14.383601 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-faddeeff-9fca-4fc9-b8a3-ec31462334b9 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:28:14.384212 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-faddeeff-9fca-4fc9-b8a3-ec31462334b9 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] [instance: 56dbfa6a-5b6f-4b82-977b-af735e6a6ac9] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:28:14.390335 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-faddeeff-9fca-4fc9-b8a3-ec31462334b9 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:28:14.390815 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-faddeeff-9fca-4fc9-b8a3-ec31462334b9 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:28:14.391253 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-faddeeff-9fca-4fc9-b8a3-ec31462334b9 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:28:14.391975 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-faddeeff-9fca-4fc9-b8a3-ec31462334b9 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:28:14.392417 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-faddeeff-9fca-4fc9-b8a3-ec31462334b9 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:28:14.392841 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-faddeeff-9fca-4fc9-b8a3-ec31462334b9 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:28:14.531599 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-faddeeff-9fca-4fc9-b8a3-ec31462334b9 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:28:14.541332 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-faddeeff-9fca-4fc9-b8a3-ec31462334b9 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:28:14.542791 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-faddeeff-9fca-4fc9-b8a3-ec31462334b9 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:28:14.543967 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-faddeeff-9fca-4fc9-b8a3-ec31462334b9 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] 10.222.0.52 "GET /compute/v2.1/servers/56dbfa6a-5b6f-4b82-977b-af735e6a6ac9" status: 200 len: 1724 microversion: 2.20 time: 0.164446 Aug 27 02:28:14.544526 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 298/596] 10.222.0.52 () {60 vars in 1277 bytes} [Sat Aug 27 02:28:14 2022] GET /compute/v2.1/servers/56dbfa6a-5b6f-4b82-977b-af735e6a6ac9 => generated 1724 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 27 02:28:14.633065 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG dbcounter [-] [120864] Writing DB stats nova_cell0:SELECT=2 {{(pid=120864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 27 02:28:14.633917 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG dbcounter [-] [120864] Writing DB stats nova_cell0:SELECT=6 {{(pid=120864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 27 02:28:14.637993 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-0fb3c4cf-c04c-4f31-b764-55858400ac91 tempest-VolumesNegativeTest-1839426864 tempest-VolumesNegativeTest-1839426864-project] Calling method '>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:28:14.638727 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-0fb3c4cf-c04c-4f31-b764-55858400ac91 tempest-VolumesNegativeTest-1839426864 tempest-VolumesNegativeTest-1839426864-project] [instance: ac07d059-1555-4f61-9a85-28396691a934] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:28:14.647713 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-0fb3c4cf-c04c-4f31-b764-55858400ac91 tempest-VolumesNegativeTest-1839426864 tempest-VolumesNegativeTest-1839426864-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:28:14.648203 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-0fb3c4cf-c04c-4f31-b764-55858400ac91 tempest-VolumesNegativeTest-1839426864 tempest-VolumesNegativeTest-1839426864-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:28:14.648639 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-0fb3c4cf-c04c-4f31-b764-55858400ac91 tempest-VolumesNegativeTest-1839426864 tempest-VolumesNegativeTest-1839426864-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:28:14.649382 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-0fb3c4cf-c04c-4f31-b764-55858400ac91 tempest-VolumesNegativeTest-1839426864 tempest-VolumesNegativeTest-1839426864-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:28:14.650199 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-0fb3c4cf-c04c-4f31-b764-55858400ac91 tempest-VolumesNegativeTest-1839426864 tempest-VolumesNegativeTest-1839426864-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:28:14.651110 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-0fb3c4cf-c04c-4f31-b764-55858400ac91 tempest-VolumesNegativeTest-1839426864 tempest-VolumesNegativeTest-1839426864-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:28:14.785660 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-0fb3c4cf-c04c-4f31-b764-55858400ac91 tempest-VolumesNegativeTest-1839426864 tempest-VolumesNegativeTest-1839426864-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '544e6c056b854bfdb9e117be686635a0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d1bead2e4f4102a53f582a4815661d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:28:14.794264 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-0fb3c4cf-c04c-4f31-b764-55858400ac91 tempest-VolumesNegativeTest-1839426864 tempest-VolumesNegativeTest-1839426864-project] 10.222.0.52 "GET /compute/v2.1/servers/ac07d059-1555-4f61-9a85-28396691a934" status: 200 len: 1314 microversion: 2.1 time: 0.162133 Aug 27 02:28:14.794830 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 299/597] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:28:14 2022] GET /compute/v2.1/servers/ac07d059-1555-4f61-9a85-28396691a934 => generated 1314 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:28:15.568307 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-0930af89-df2e-4bca-b818-8a4e50f05a09 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:28:15.568957 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-0930af89-df2e-4bca-b818-8a4e50f05a09 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] [instance: 56dbfa6a-5b6f-4b82-977b-af735e6a6ac9] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:28:15.570782 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG dbcounter [-] [120863] Writing DB stats nova_cell0:SELECT=2 {{(pid=120863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 27 02:28:15.576391 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-0930af89-df2e-4bca-b818-8a4e50f05a09 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:28:15.576877 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-0930af89-df2e-4bca-b818-8a4e50f05a09 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:28:15.577343 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-0930af89-df2e-4bca-b818-8a4e50f05a09 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:28:15.578042 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-0930af89-df2e-4bca-b818-8a4e50f05a09 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:28:15.578495 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-0930af89-df2e-4bca-b818-8a4e50f05a09 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:28:15.578918 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-0930af89-df2e-4bca-b818-8a4e50f05a09 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:28:15.724809 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-0930af89-df2e-4bca-b818-8a4e50f05a09 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:28:15.733632 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-0930af89-df2e-4bca-b818-8a4e50f05a09 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:28:15.735214 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-0930af89-df2e-4bca-b818-8a4e50f05a09 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:28:15.736398 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-0930af89-df2e-4bca-b818-8a4e50f05a09 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] 10.222.0.52 "GET /compute/v2.1/servers/56dbfa6a-5b6f-4b82-977b-af735e6a6ac9" status: 200 len: 1724 microversion: 2.20 time: 0.172190 Aug 27 02:28:15.736958 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 299/598] 10.222.0.52 () {60 vars in 1277 bytes} [Sat Aug 27 02:28:15 2022] GET /compute/v2.1/servers/56dbfa6a-5b6f-4b82-977b-af735e6a6ac9 => generated 1724 bytes in 173 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 27 02:28:15.815486 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-7bcf0436-f1c5-4c29-866f-b7fe78a8d631 tempest-VolumesNegativeTest-1839426864 tempest-VolumesNegativeTest-1839426864-project] Calling method '>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:28:15.816092 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-7bcf0436-f1c5-4c29-866f-b7fe78a8d631 tempest-VolumesNegativeTest-1839426864 tempest-VolumesNegativeTest-1839426864-project] [instance: ac07d059-1555-4f61-9a85-28396691a934] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:28:15.823108 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-7bcf0436-f1c5-4c29-866f-b7fe78a8d631 tempest-VolumesNegativeTest-1839426864 tempest-VolumesNegativeTest-1839426864-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:28:15.823582 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-7bcf0436-f1c5-4c29-866f-b7fe78a8d631 tempest-VolumesNegativeTest-1839426864 tempest-VolumesNegativeTest-1839426864-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:28:15.823995 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-7bcf0436-f1c5-4c29-866f-b7fe78a8d631 tempest-VolumesNegativeTest-1839426864 tempest-VolumesNegativeTest-1839426864-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:28:15.824670 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-7bcf0436-f1c5-4c29-866f-b7fe78a8d631 tempest-VolumesNegativeTest-1839426864 tempest-VolumesNegativeTest-1839426864-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:28:15.825135 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-7bcf0436-f1c5-4c29-866f-b7fe78a8d631 tempest-VolumesNegativeTest-1839426864 tempest-VolumesNegativeTest-1839426864-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:28:15.825666 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-7bcf0436-f1c5-4c29-866f-b7fe78a8d631 tempest-VolumesNegativeTest-1839426864 tempest-VolumesNegativeTest-1839426864-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:28:15.965975 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-7bcf0436-f1c5-4c29-866f-b7fe78a8d631 tempest-VolumesNegativeTest-1839426864 tempest-VolumesNegativeTest-1839426864-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '544e6c056b854bfdb9e117be686635a0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d1bead2e4f4102a53f582a4815661d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:28:15.976131 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-7bcf0436-f1c5-4c29-866f-b7fe78a8d631 tempest-VolumesNegativeTest-1839426864 tempest-VolumesNegativeTest-1839426864-project] 10.222.0.52 "GET /compute/v2.1/servers/ac07d059-1555-4f61-9a85-28396691a934" status: 200 len: 1314 microversion: 2.1 time: 0.164333 Aug 27 02:28:15.977112 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 300/599] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:28:15 2022] GET /compute/v2.1/servers/ac07d059-1555-4f61-9a85-28396691a934 => generated 1314 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:28:16.762564 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-6892b3b7-0273-431a-b19f-1a6b61edbe8b tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:28:16.763197 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-6892b3b7-0273-431a-b19f-1a6b61edbe8b tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] [instance: 56dbfa6a-5b6f-4b82-977b-af735e6a6ac9] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:28:16.769837 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-6892b3b7-0273-431a-b19f-1a6b61edbe8b tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:28:16.770420 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-6892b3b7-0273-431a-b19f-1a6b61edbe8b tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:28:16.770995 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-6892b3b7-0273-431a-b19f-1a6b61edbe8b tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:28:16.771774 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-6892b3b7-0273-431a-b19f-1a6b61edbe8b tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:28:16.772217 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-6892b3b7-0273-431a-b19f-1a6b61edbe8b tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:28:16.772637 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-6892b3b7-0273-431a-b19f-1a6b61edbe8b tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:28:16.914941 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-6892b3b7-0273-431a-b19f-1a6b61edbe8b tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:28:16.923262 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-6892b3b7-0273-431a-b19f-1a6b61edbe8b tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:28:16.924691 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-6892b3b7-0273-431a-b19f-1a6b61edbe8b tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:28:16.925876 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-6892b3b7-0273-431a-b19f-1a6b61edbe8b tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] 10.222.0.52 "GET /compute/v2.1/servers/56dbfa6a-5b6f-4b82-977b-af735e6a6ac9" status: 200 len: 1724 microversion: 2.20 time: 0.167375 Aug 27 02:28:16.926404 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 300/600] 10.222.0.52 () {60 vars in 1277 bytes} [Sat Aug 27 02:28:16 2022] GET /compute/v2.1/servers/56dbfa6a-5b6f-4b82-977b-af735e6a6ac9 => generated 1724 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 27 02:28:17.003265 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-20a30e4c-ce03-4b90-a358-e8ca020704bd tempest-VolumesNegativeTest-1839426864 tempest-VolumesNegativeTest-1839426864-project] Calling method '>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:28:17.003893 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-20a30e4c-ce03-4b90-a358-e8ca020704bd tempest-VolumesNegativeTest-1839426864 tempest-VolumesNegativeTest-1839426864-project] [instance: ac07d059-1555-4f61-9a85-28396691a934] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:28:17.011241 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-20a30e4c-ce03-4b90-a358-e8ca020704bd tempest-VolumesNegativeTest-1839426864 tempest-VolumesNegativeTest-1839426864-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:28:17.011724 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-20a30e4c-ce03-4b90-a358-e8ca020704bd tempest-VolumesNegativeTest-1839426864 tempest-VolumesNegativeTest-1839426864-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:28:17.012168 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-20a30e4c-ce03-4b90-a358-e8ca020704bd tempest-VolumesNegativeTest-1839426864 tempest-VolumesNegativeTest-1839426864-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:28:17.012877 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-20a30e4c-ce03-4b90-a358-e8ca020704bd tempest-VolumesNegativeTest-1839426864 tempest-VolumesNegativeTest-1839426864-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:28:17.013660 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-20a30e4c-ce03-4b90-a358-e8ca020704bd tempest-VolumesNegativeTest-1839426864 tempest-VolumesNegativeTest-1839426864-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:28:17.014848 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-20a30e4c-ce03-4b90-a358-e8ca020704bd tempest-VolumesNegativeTest-1839426864 tempest-VolumesNegativeTest-1839426864-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:28:17.165974 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-20a30e4c-ce03-4b90-a358-e8ca020704bd tempest-VolumesNegativeTest-1839426864 tempest-VolumesNegativeTest-1839426864-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '544e6c056b854bfdb9e117be686635a0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d1bead2e4f4102a53f582a4815661d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:28:17.174621 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-20a30e4c-ce03-4b90-a358-e8ca020704bd tempest-VolumesNegativeTest-1839426864 tempest-VolumesNegativeTest-1839426864-project] 10.222.0.52 "GET /compute/v2.1/servers/ac07d059-1555-4f61-9a85-28396691a934" status: 200 len: 1314 microversion: 2.1 time: 0.175273 Aug 27 02:28:17.175302 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 301/601] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:28:16 2022] GET /compute/v2.1/servers/ac07d059-1555-4f61-9a85-28396691a934 => generated 1314 bytes in 177 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:28:17.950347 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-c7d831d2-898d-456a-aa94-b59655d32759 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:28:17.950979 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-c7d831d2-898d-456a-aa94-b59655d32759 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] [instance: 56dbfa6a-5b6f-4b82-977b-af735e6a6ac9] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:28:17.958764 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-c7d831d2-898d-456a-aa94-b59655d32759 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:28:17.959293 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-c7d831d2-898d-456a-aa94-b59655d32759 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:28:17.959795 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-c7d831d2-898d-456a-aa94-b59655d32759 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:28:17.960549 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-c7d831d2-898d-456a-aa94-b59655d32759 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:28:17.961030 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-c7d831d2-898d-456a-aa94-b59655d32759 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:28:17.961546 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-c7d831d2-898d-456a-aa94-b59655d32759 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:28:18.098796 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-c7d831d2-898d-456a-aa94-b59655d32759 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:28:18.107633 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-c7d831d2-898d-456a-aa94-b59655d32759 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:28:18.109080 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-c7d831d2-898d-456a-aa94-b59655d32759 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:28:18.110271 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-c7d831d2-898d-456a-aa94-b59655d32759 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] 10.222.0.52 "GET /compute/v2.1/servers/56dbfa6a-5b6f-4b82-977b-af735e6a6ac9" status: 200 len: 1724 microversion: 2.20 time: 0.164420 Aug 27 02:28:18.110799 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 301/602] 10.222.0.52 () {60 vars in 1277 bytes} [Sat Aug 27 02:28:17 2022] GET /compute/v2.1/servers/56dbfa6a-5b6f-4b82-977b-af735e6a6ac9 => generated 1724 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 27 02:28:18.196640 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-f62f2754-5d1f-4bd9-8e97-71254d4d33eb tempest-VolumesNegativeTest-1839426864 tempest-VolumesNegativeTest-1839426864-project] Calling method '>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:28:18.197443 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-f62f2754-5d1f-4bd9-8e97-71254d4d33eb tempest-VolumesNegativeTest-1839426864 tempest-VolumesNegativeTest-1839426864-project] [instance: ac07d059-1555-4f61-9a85-28396691a934] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:28:18.206439 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-f62f2754-5d1f-4bd9-8e97-71254d4d33eb tempest-VolumesNegativeTest-1839426864 tempest-VolumesNegativeTest-1839426864-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:28:18.211914 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-f62f2754-5d1f-4bd9-8e97-71254d4d33eb tempest-VolumesNegativeTest-1839426864 tempest-VolumesNegativeTest-1839426864-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:28:18.211914 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-f62f2754-5d1f-4bd9-8e97-71254d4d33eb tempest-VolumesNegativeTest-1839426864 tempest-VolumesNegativeTest-1839426864-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:28:18.211914 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-f62f2754-5d1f-4bd9-8e97-71254d4d33eb tempest-VolumesNegativeTest-1839426864 tempest-VolumesNegativeTest-1839426864-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:28:18.211914 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-f62f2754-5d1f-4bd9-8e97-71254d4d33eb tempest-VolumesNegativeTest-1839426864 tempest-VolumesNegativeTest-1839426864-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:28:18.212341 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-f62f2754-5d1f-4bd9-8e97-71254d4d33eb tempest-VolumesNegativeTest-1839426864 tempest-VolumesNegativeTest-1839426864-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:28:18.366228 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-f62f2754-5d1f-4bd9-8e97-71254d4d33eb tempest-VolumesNegativeTest-1839426864 tempest-VolumesNegativeTest-1839426864-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '544e6c056b854bfdb9e117be686635a0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d1bead2e4f4102a53f582a4815661d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:28:18.374205 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-f62f2754-5d1f-4bd9-8e97-71254d4d33eb tempest-VolumesNegativeTest-1839426864 tempest-VolumesNegativeTest-1839426864-project] 10.222.0.52 "GET /compute/v2.1/servers/ac07d059-1555-4f61-9a85-28396691a934" status: 200 len: 1314 microversion: 2.1 time: 0.181968 Aug 27 02:28:18.374831 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 302/603] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:28:18 2022] GET /compute/v2.1/servers/ac07d059-1555-4f61-9a85-28396691a934 => generated 1314 bytes in 183 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 27 02:28:19.134299 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-73ddf7b3-6a95-4f02-9e38-1e39e2567943 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:28:19.134929 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-73ddf7b3-6a95-4f02-9e38-1e39e2567943 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] [instance: 56dbfa6a-5b6f-4b82-977b-af735e6a6ac9] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:28:19.141509 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-73ddf7b3-6a95-4f02-9e38-1e39e2567943 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:28:19.141993 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-73ddf7b3-6a95-4f02-9e38-1e39e2567943 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:28:19.142595 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-73ddf7b3-6a95-4f02-9e38-1e39e2567943 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:28:19.143297 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-73ddf7b3-6a95-4f02-9e38-1e39e2567943 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:28:19.143743 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-73ddf7b3-6a95-4f02-9e38-1e39e2567943 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:28:19.144165 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-73ddf7b3-6a95-4f02-9e38-1e39e2567943 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:28:19.290764 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-73ddf7b3-6a95-4f02-9e38-1e39e2567943 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:28:19.299411 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-73ddf7b3-6a95-4f02-9e38-1e39e2567943 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:28:19.300845 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-73ddf7b3-6a95-4f02-9e38-1e39e2567943 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:28:19.302024 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-73ddf7b3-6a95-4f02-9e38-1e39e2567943 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] 10.222.0.52 "GET /compute/v2.1/servers/56dbfa6a-5b6f-4b82-977b-af735e6a6ac9" status: 200 len: 1724 microversion: 2.20 time: 0.171734 Aug 27 02:28:19.302657 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 302/604] 10.222.0.52 () {60 vars in 1277 bytes} [Sat Aug 27 02:28:19 2022] GET /compute/v2.1/servers/56dbfa6a-5b6f-4b82-977b-af735e6a6ac9 => generated 1724 bytes in 173 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Aug 27 02:28:19.397869 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-d38a65cb-2c8c-4d13-8044-d68dbbb1230c tempest-VolumesNegativeTest-1839426864 tempest-VolumesNegativeTest-1839426864-project] Calling method '>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:28:19.398441 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-d38a65cb-2c8c-4d13-8044-d68dbbb1230c tempest-VolumesNegativeTest-1839426864 tempest-VolumesNegativeTest-1839426864-project] [instance: ac07d059-1555-4f61-9a85-28396691a934] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:28:19.405253 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-d38a65cb-2c8c-4d13-8044-d68dbbb1230c tempest-VolumesNegativeTest-1839426864 tempest-VolumesNegativeTest-1839426864-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:28:19.405712 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-d38a65cb-2c8c-4d13-8044-d68dbbb1230c tempest-VolumesNegativeTest-1839426864 tempest-VolumesNegativeTest-1839426864-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:28:19.406143 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-d38a65cb-2c8c-4d13-8044-d68dbbb1230c tempest-VolumesNegativeTest-1839426864 tempest-VolumesNegativeTest-1839426864-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:28:19.406843 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-d38a65cb-2c8c-4d13-8044-d68dbbb1230c tempest-VolumesNegativeTest-1839426864 tempest-VolumesNegativeTest-1839426864-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:28:19.407300 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-d38a65cb-2c8c-4d13-8044-d68dbbb1230c tempest-VolumesNegativeTest-1839426864 tempest-VolumesNegativeTest-1839426864-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:28:19.407713 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-d38a65cb-2c8c-4d13-8044-d68dbbb1230c tempest-VolumesNegativeTest-1839426864 tempest-VolumesNegativeTest-1839426864-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:28:19.556029 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-d38a65cb-2c8c-4d13-8044-d68dbbb1230c tempest-VolumesNegativeTest-1839426864 tempest-VolumesNegativeTest-1839426864-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '544e6c056b854bfdb9e117be686635a0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d1bead2e4f4102a53f582a4815661d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:28:19.564521 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-d38a65cb-2c8c-4d13-8044-d68dbbb1230c tempest-VolumesNegativeTest-1839426864 tempest-VolumesNegativeTest-1839426864-project] 10.222.0.52 "GET /compute/v2.1/servers/ac07d059-1555-4f61-9a85-28396691a934" status: 200 len: 1314 microversion: 2.1 time: 0.170688 Aug 27 02:28:19.565095 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 303/605] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:28:19 2022] GET /compute/v2.1/servers/ac07d059-1555-4f61-9a85-28396691a934 => generated 1314 bytes in 172 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:28:20.328104 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-8ccdbbd0-a3ef-46e4-8925-2fa8e71af2e2 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:28:20.328669 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-8ccdbbd0-a3ef-46e4-8925-2fa8e71af2e2 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] [instance: 56dbfa6a-5b6f-4b82-977b-af735e6a6ac9] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:28:20.337357 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-8ccdbbd0-a3ef-46e4-8925-2fa8e71af2e2 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:28:20.337841 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-8ccdbbd0-a3ef-46e4-8925-2fa8e71af2e2 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:28:20.338314 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-8ccdbbd0-a3ef-46e4-8925-2fa8e71af2e2 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:28:20.339042 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-8ccdbbd0-a3ef-46e4-8925-2fa8e71af2e2 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:28:20.339496 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-8ccdbbd0-a3ef-46e4-8925-2fa8e71af2e2 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:28:20.339923 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-8ccdbbd0-a3ef-46e4-8925-2fa8e71af2e2 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:28:20.489456 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-8ccdbbd0-a3ef-46e4-8925-2fa8e71af2e2 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:28:20.497967 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-8ccdbbd0-a3ef-46e4-8925-2fa8e71af2e2 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:28:20.499415 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-8ccdbbd0-a3ef-46e4-8925-2fa8e71af2e2 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:28:20.500582 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-8ccdbbd0-a3ef-46e4-8925-2fa8e71af2e2 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] 10.222.0.52 "GET /compute/v2.1/servers/56dbfa6a-5b6f-4b82-977b-af735e6a6ac9" status: 200 len: 1724 microversion: 2.20 time: 0.176465 Aug 27 02:28:20.501164 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 303/606] 10.222.0.52 () {60 vars in 1277 bytes} [Sat Aug 27 02:28:20 2022] GET /compute/v2.1/servers/56dbfa6a-5b6f-4b82-977b-af735e6a6ac9 => generated 1724 bytes in 178 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 27 02:28:20.593681 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-37baac66-5b8e-4aac-8188-a9ce812122f0 tempest-VolumesNegativeTest-1839426864 tempest-VolumesNegativeTest-1839426864-project] Calling method '>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:28:20.594773 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-37baac66-5b8e-4aac-8188-a9ce812122f0 tempest-VolumesNegativeTest-1839426864 tempest-VolumesNegativeTest-1839426864-project] [instance: ac07d059-1555-4f61-9a85-28396691a934] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:28:20.603891 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-37baac66-5b8e-4aac-8188-a9ce812122f0 tempest-VolumesNegativeTest-1839426864 tempest-VolumesNegativeTest-1839426864-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:28:20.604382 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-37baac66-5b8e-4aac-8188-a9ce812122f0 tempest-VolumesNegativeTest-1839426864 tempest-VolumesNegativeTest-1839426864-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:28:20.604811 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-37baac66-5b8e-4aac-8188-a9ce812122f0 tempest-VolumesNegativeTest-1839426864 tempest-VolumesNegativeTest-1839426864-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:28:20.605546 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-37baac66-5b8e-4aac-8188-a9ce812122f0 tempest-VolumesNegativeTest-1839426864 tempest-VolumesNegativeTest-1839426864-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:28:20.605993 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-37baac66-5b8e-4aac-8188-a9ce812122f0 tempest-VolumesNegativeTest-1839426864 tempest-VolumesNegativeTest-1839426864-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:28:20.606409 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-37baac66-5b8e-4aac-8188-a9ce812122f0 tempest-VolumesNegativeTest-1839426864 tempest-VolumesNegativeTest-1839426864-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:28:20.749819 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-37baac66-5b8e-4aac-8188-a9ce812122f0 tempest-VolumesNegativeTest-1839426864 tempest-VolumesNegativeTest-1839426864-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '544e6c056b854bfdb9e117be686635a0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d1bead2e4f4102a53f582a4815661d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:28:20.758279 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-37baac66-5b8e-4aac-8188-a9ce812122f0 tempest-VolumesNegativeTest-1839426864 tempest-VolumesNegativeTest-1839426864-project] 10.222.0.52 "GET /compute/v2.1/servers/ac07d059-1555-4f61-9a85-28396691a934" status: 200 len: 1314 microversion: 2.1 time: 0.170622 Aug 27 02:28:20.758860 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 304/607] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:28:20 2022] GET /compute/v2.1/servers/ac07d059-1555-4f61-9a85-28396691a934 => generated 1314 bytes in 172 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 27 02:28:21.510611 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-f35f85a2-5c5a-4db2-b3f1-e9221d399011 tempest-VolumesSnapshotTestJSON-1750456804 tempest-VolumesSnapshotTestJSON-1750456804-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:28:21.511184 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-f35f85a2-5c5a-4db2-b3f1-e9221d399011 tempest-VolumesSnapshotTestJSON-1750456804 tempest-VolumesSnapshotTestJSON-1750456804-project] [instance: 46dae88e-3429-47c7-9376-c4c0d9cddd29] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:28:21.518318 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-f35f85a2-5c5a-4db2-b3f1-e9221d399011 tempest-VolumesSnapshotTestJSON-1750456804 tempest-VolumesSnapshotTestJSON-1750456804-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:28:21.518318 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-f35f85a2-5c5a-4db2-b3f1-e9221d399011 tempest-VolumesSnapshotTestJSON-1750456804 tempest-VolumesSnapshotTestJSON-1750456804-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:28:21.518597 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-f35f85a2-5c5a-4db2-b3f1-e9221d399011 tempest-VolumesSnapshotTestJSON-1750456804 tempest-VolumesSnapshotTestJSON-1750456804-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:28:21.519397 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-f35f85a2-5c5a-4db2-b3f1-e9221d399011 tempest-VolumesSnapshotTestJSON-1750456804 tempest-VolumesSnapshotTestJSON-1750456804-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:28:21.519979 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-f35f85a2-5c5a-4db2-b3f1-e9221d399011 tempest-VolumesSnapshotTestJSON-1750456804 tempest-VolumesSnapshotTestJSON-1750456804-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:28:21.520658 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-f35f85a2-5c5a-4db2-b3f1-e9221d399011 tempest-VolumesSnapshotTestJSON-1750456804 tempest-VolumesSnapshotTestJSON-1750456804-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:28:21.524594 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-1eee57e7-0739-4d81-87fe-28b9944c1b80 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Calling method '>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:28:21.525327 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-1eee57e7-0739-4d81-87fe-28b9944c1b80 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] [instance: 56dbfa6a-5b6f-4b82-977b-af735e6a6ac9] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:28:21.531804 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-1eee57e7-0739-4d81-87fe-28b9944c1b80 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:28:21.532597 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-1eee57e7-0739-4d81-87fe-28b9944c1b80 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:28:21.533342 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-1eee57e7-0739-4d81-87fe-28b9944c1b80 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:28:21.534283 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-1eee57e7-0739-4d81-87fe-28b9944c1b80 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:28:21.534835 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-1eee57e7-0739-4d81-87fe-28b9944c1b80 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:28:21.535322 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-1eee57e7-0739-4d81-87fe-28b9944c1b80 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:28:21.652543 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.objects.instance [None req-f35f85a2-5c5a-4db2-b3f1-e9221d399011 tempest-VolumesSnapshotTestJSON-1750456804 tempest-VolumesSnapshotTestJSON-1750456804-project] Lazy-loading 'services' on Instance uuid 46dae88e-3429-47c7-9376-c4c0d9cddd29 {{(pid=120863) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Aug 27 02:28:21.690169 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-1eee57e7-0739-4d81-87fe-28b9944c1b80 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:28:21.699401 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-1eee57e7-0739-4d81-87fe-28b9944c1b80 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:28:21.701892 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-1eee57e7-0739-4d81-87fe-28b9944c1b80 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:28:21.703342 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-1eee57e7-0739-4d81-87fe-28b9944c1b80 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] 10.222.0.52 "GET /compute/v2.1/servers/56dbfa6a-5b6f-4b82-977b-af735e6a6ac9" status: 200 len: 1724 microversion: 2.20 time: 0.183023 Aug 27 02:28:21.718377 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 305/608] 10.222.0.52 () {60 vars in 1277 bytes} [Sat Aug 27 02:28:21 2022] GET /compute/v2.1/servers/56dbfa6a-5b6f-4b82-977b-af735e6a6ac9 => generated 1724 bytes in 199 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 27 02:28:21.778904 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-9558df2f-3cd7-48f8-bf07-0c2cf3ce8808 tempest-VolumesNegativeTest-1839426864 tempest-VolumesNegativeTest-1839426864-project] Calling method '>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:28:21.779517 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-9558df2f-3cd7-48f8-bf07-0c2cf3ce8808 tempest-VolumesNegativeTest-1839426864 tempest-VolumesNegativeTest-1839426864-project] [instance: ac07d059-1555-4f61-9a85-28396691a934] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:28:21.788982 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-9558df2f-3cd7-48f8-bf07-0c2cf3ce8808 tempest-VolumesNegativeTest-1839426864 tempest-VolumesNegativeTest-1839426864-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:28:21.789509 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-9558df2f-3cd7-48f8-bf07-0c2cf3ce8808 tempest-VolumesNegativeTest-1839426864 tempest-VolumesNegativeTest-1839426864-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:28:21.789942 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-9558df2f-3cd7-48f8-bf07-0c2cf3ce8808 tempest-VolumesNegativeTest-1839426864 tempest-VolumesNegativeTest-1839426864-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:28:21.790661 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-9558df2f-3cd7-48f8-bf07-0c2cf3ce8808 tempest-VolumesNegativeTest-1839426864 tempest-VolumesNegativeTest-1839426864-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:28:21.791392 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-9558df2f-3cd7-48f8-bf07-0c2cf3ce8808 tempest-VolumesNegativeTest-1839426864 tempest-VolumesNegativeTest-1839426864-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:28:21.791901 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-9558df2f-3cd7-48f8-bf07-0c2cf3ce8808 tempest-VolumesNegativeTest-1839426864 tempest-VolumesNegativeTest-1839426864-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:28:21.827999 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-f35f85a2-5c5a-4db2-b3f1-e9221d399011 tempest-VolumesSnapshotTestJSON-1750456804 tempest-VolumesSnapshotTestJSON-1750456804-project] 10.222.0.52 "DELETE /compute/v2.1/servers/46dae88e-3429-47c7-9376-c4c0d9cddd29/os-volume_attachments/35da299d-3a4d-4a2b-a065-2c4fcbbdef52" status: 202 len: 0 microversion: 2.1 time: 0.321647 Aug 27 02:28:21.828604 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 304/609] 10.222.0.52 () {58 vars in 1416 bytes} [Sat Aug 27 02:28:21 2022] DELETE /compute/v2.1/servers/46dae88e-3429-47c7-9376-c4c0d9cddd29/os-volume_attachments/35da299d-3a4d-4a2b-a065-2c4fcbbdef52 => generated 0 bytes in 324 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Aug 27 02:28:21.967879 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-9558df2f-3cd7-48f8-bf07-0c2cf3ce8808 tempest-VolumesNegativeTest-1839426864 tempest-VolumesNegativeTest-1839426864-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '544e6c056b854bfdb9e117be686635a0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d1bead2e4f4102a53f582a4815661d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:28:21.978804 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-9558df2f-3cd7-48f8-bf07-0c2cf3ce8808 tempest-VolumesNegativeTest-1839426864 tempest-VolumesNegativeTest-1839426864-project] 10.222.0.52 "GET /compute/v2.1/servers/ac07d059-1555-4f61-9a85-28396691a934" status: 200 len: 1314 microversion: 2.1 time: 0.203551 Aug 27 02:28:21.990102 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 306/610] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:28:21 2022] GET /compute/v2.1/servers/ac07d059-1555-4f61-9a85-28396691a934 => generated 1314 bytes in 210 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 27 02:28:22.729238 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-8489c08d-ff98-4437-ab49-4d950a36bcec tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:28:22.729936 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-8489c08d-ff98-4437-ab49-4d950a36bcec tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] [instance: 56dbfa6a-5b6f-4b82-977b-af735e6a6ac9] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:28:22.736998 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-8489c08d-ff98-4437-ab49-4d950a36bcec tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:28:22.737524 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-8489c08d-ff98-4437-ab49-4d950a36bcec tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:28:22.737957 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-8489c08d-ff98-4437-ab49-4d950a36bcec tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:28:22.738653 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-8489c08d-ff98-4437-ab49-4d950a36bcec tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:28:22.739101 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-8489c08d-ff98-4437-ab49-4d950a36bcec tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:28:22.739526 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-8489c08d-ff98-4437-ab49-4d950a36bcec tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:28:22.891299 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-8489c08d-ff98-4437-ab49-4d950a36bcec tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:28:22.900221 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-8489c08d-ff98-4437-ab49-4d950a36bcec tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:28:22.901695 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-8489c08d-ff98-4437-ab49-4d950a36bcec tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:28:22.902866 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-8489c08d-ff98-4437-ab49-4d950a36bcec tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] 10.222.0.52 "GET /compute/v2.1/servers/56dbfa6a-5b6f-4b82-977b-af735e6a6ac9" status: 200 len: 1724 microversion: 2.20 time: 0.178554 Aug 27 02:28:22.903428 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 305/611] 10.222.0.52 () {60 vars in 1277 bytes} [Sat Aug 27 02:28:22 2022] GET /compute/v2.1/servers/56dbfa6a-5b6f-4b82-977b-af735e6a6ac9 => generated 1724 bytes in 180 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 27 02:28:23.001377 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-c45cf44f-e3c6-4e7a-99af-621104b1a635 tempest-VolumesNegativeTest-1839426864 tempest-VolumesNegativeTest-1839426864-project] Calling method '>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:28:23.002213 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-c45cf44f-e3c6-4e7a-99af-621104b1a635 tempest-VolumesNegativeTest-1839426864 tempest-VolumesNegativeTest-1839426864-project] [instance: ac07d059-1555-4f61-9a85-28396691a934] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:28:23.009579 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-c45cf44f-e3c6-4e7a-99af-621104b1a635 tempest-VolumesNegativeTest-1839426864 tempest-VolumesNegativeTest-1839426864-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:28:23.010068 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-c45cf44f-e3c6-4e7a-99af-621104b1a635 tempest-VolumesNegativeTest-1839426864 tempest-VolumesNegativeTest-1839426864-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:28:23.010494 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-c45cf44f-e3c6-4e7a-99af-621104b1a635 tempest-VolumesNegativeTest-1839426864 tempest-VolumesNegativeTest-1839426864-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:28:23.011208 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-c45cf44f-e3c6-4e7a-99af-621104b1a635 tempest-VolumesNegativeTest-1839426864 tempest-VolumesNegativeTest-1839426864-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:28:23.011682 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-c45cf44f-e3c6-4e7a-99af-621104b1a635 tempest-VolumesNegativeTest-1839426864 tempest-VolumesNegativeTest-1839426864-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:28:23.012127 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-c45cf44f-e3c6-4e7a-99af-621104b1a635 tempest-VolumesNegativeTest-1839426864 tempest-VolumesNegativeTest-1839426864-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:28:23.158124 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-c45cf44f-e3c6-4e7a-99af-621104b1a635 tempest-VolumesNegativeTest-1839426864 tempest-VolumesNegativeTest-1839426864-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '544e6c056b854bfdb9e117be686635a0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d1bead2e4f4102a53f582a4815661d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:28:23.167122 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-c45cf44f-e3c6-4e7a-99af-621104b1a635 tempest-VolumesNegativeTest-1839426864 tempest-VolumesNegativeTest-1839426864-project] 10.222.0.52 "GET /compute/v2.1/servers/ac07d059-1555-4f61-9a85-28396691a934" status: 200 len: 1314 microversion: 2.1 time: 0.169838 Aug 27 02:28:23.167686 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 307/612] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:28:22 2022] GET /compute/v2.1/servers/ac07d059-1555-4f61-9a85-28396691a934 => generated 1314 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 27 02:28:23.344587 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [req-18fab081-4bf2-4e55-9b0b-e11e97911344 req-308903c2-37ab-4c05-9575-9550e920c70e service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "ac07d059-1555-4f61-9a85-28396691a934", "name": "network-vif-plugged", "status": "completed", "tag": "42163688-0f26-433a-b60b-97256fc0be15"}]} {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 27 02:28:23.355421 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [req-18fab081-4bf2-4e55-9b0b-e11e97911344 req-308903c2-37ab-4c05-9575-9550e920c70e service nova] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:28:23.356058 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [req-18fab081-4bf2-4e55-9b0b-e11e97911344 req-308903c2-37ab-4c05-9575-9550e920c70e service nova] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:28:23.356481 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [req-18fab081-4bf2-4e55-9b0b-e11e97911344 req-308903c2-37ab-4c05-9575-9550e920c70e service nova] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:28:23.378071 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.compute.server_external_events [req-18fab081-4bf2-4e55-9b0b-e11e97911344 req-308903c2-37ab-4c05-9575-9550e920c70e service nova] Creating event network-vif-plugged:42163688-0f26-433a-b60b-97256fc0be15 for instance ac07d059-1555-4f61-9a85-28396691a934 on hv-cinder-84630 Aug 27 02:28:23.389135 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [req-18fab081-4bf2-4e55-9b0b-e11e97911344 req-308903c2-37ab-4c05-9575-9550e920c70e service nova] 10.222.0.52 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.045141 Aug 27 02:28:23.390224 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 306/613] 10.222.0.52 () {64 vars in 1317 bytes} [Sat Aug 27 02:28:23 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 51 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 27 02:28:23.928437 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-b1fc7759-d490-44ba-9554-50bb16568181 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Calling method '>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:28:23.929150 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-b1fc7759-d490-44ba-9554-50bb16568181 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] [instance: 56dbfa6a-5b6f-4b82-977b-af735e6a6ac9] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:28:23.935837 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-b1fc7759-d490-44ba-9554-50bb16568181 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:28:23.936321 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-b1fc7759-d490-44ba-9554-50bb16568181 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:28:23.936768 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-b1fc7759-d490-44ba-9554-50bb16568181 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:28:23.937507 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-b1fc7759-d490-44ba-9554-50bb16568181 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:28:23.937965 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-b1fc7759-d490-44ba-9554-50bb16568181 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:28:23.938391 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-b1fc7759-d490-44ba-9554-50bb16568181 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:28:24.055610 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-389bab9c-058c-44bc-8395-2bb3966d7735 tempest-VolumesSnapshotTestJSON-1750456804 tempest-VolumesSnapshotTestJSON-1750456804-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:28:24.056215 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-389bab9c-058c-44bc-8395-2bb3966d7735 tempest-VolumesSnapshotTestJSON-1750456804 tempest-VolumesSnapshotTestJSON-1750456804-project] [instance: 46dae88e-3429-47c7-9376-c4c0d9cddd29] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:28:24.062561 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-389bab9c-058c-44bc-8395-2bb3966d7735 tempest-VolumesSnapshotTestJSON-1750456804 tempest-VolumesSnapshotTestJSON-1750456804-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:28:24.063074 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-389bab9c-058c-44bc-8395-2bb3966d7735 tempest-VolumesSnapshotTestJSON-1750456804 tempest-VolumesSnapshotTestJSON-1750456804-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:28:24.063553 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-389bab9c-058c-44bc-8395-2bb3966d7735 tempest-VolumesSnapshotTestJSON-1750456804 tempest-VolumesSnapshotTestJSON-1750456804-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:28:24.064318 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-389bab9c-058c-44bc-8395-2bb3966d7735 tempest-VolumesSnapshotTestJSON-1750456804 tempest-VolumesSnapshotTestJSON-1750456804-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:28:24.064782 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-389bab9c-058c-44bc-8395-2bb3966d7735 tempest-VolumesSnapshotTestJSON-1750456804 tempest-VolumesSnapshotTestJSON-1750456804-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:28:24.065246 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-389bab9c-058c-44bc-8395-2bb3966d7735 tempest-VolumesSnapshotTestJSON-1750456804 tempest-VolumesSnapshotTestJSON-1750456804-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:28:24.086881 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-b1fc7759-d490-44ba-9554-50bb16568181 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:28:24.096081 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-389bab9c-058c-44bc-8395-2bb3966d7735 tempest-VolumesSnapshotTestJSON-1750456804 tempest-VolumesSnapshotTestJSON-1750456804-project] [instance: 46dae88e-3429-47c7-9376-c4c0d9cddd29] Going to try to terminate instance {{(pid=120863) delete /opt/stack/nova/nova/compute/api.py:2719}} Aug 27 02:28:24.101353 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-b1fc7759-d490-44ba-9554-50bb16568181 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:28:24.102812 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-b1fc7759-d490-44ba-9554-50bb16568181 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:28:24.104009 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-b1fc7759-d490-44ba-9554-50bb16568181 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] 10.222.0.52 "GET /compute/v2.1/servers/56dbfa6a-5b6f-4b82-977b-af735e6a6ac9" status: 200 len: 1724 microversion: 2.20 time: 0.180002 Aug 27 02:28:24.104565 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 308/614] 10.222.0.52 () {60 vars in 1277 bytes} [Sat Aug 27 02:28:23 2022] GET /compute/v2.1/servers/56dbfa6a-5b6f-4b82-977b-af735e6a6ac9 => generated 1724 bytes in 181 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 27 02:28:24.176150 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-389bab9c-058c-44bc-8395-2bb3966d7735 tempest-VolumesSnapshotTestJSON-1750456804 tempest-VolumesSnapshotTestJSON-1750456804-project] 10.222.0.52 "DELETE /compute/v2.1/servers/46dae88e-3429-47c7-9376-c4c0d9cddd29" status: 204 len: 0 microversion: 2.1 time: 0.125151 Aug 27 02:28:24.176937 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 307/615] 10.222.0.52 () {58 vars in 1239 bytes} [Sat Aug 27 02:28:24 2022] DELETE /compute/v2.1/servers/46dae88e-3429-47c7-9376-c4c0d9cddd29 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Aug 27 02:28:24.185320 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-36681c3a-a271-4c76-88b6-03a2fa86930a tempest-VolumesSnapshotTestJSON-1750456804 tempest-VolumesSnapshotTestJSON-1750456804-project] Calling method '>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:28:24.185931 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-36681c3a-a271-4c76-88b6-03a2fa86930a tempest-VolumesSnapshotTestJSON-1750456804 tempest-VolumesSnapshotTestJSON-1750456804-project] [instance: 46dae88e-3429-47c7-9376-c4c0d9cddd29] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:28:24.188484 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-e84240a7-30a7-485e-8cce-0e248c8c2409 tempest-VolumesNegativeTest-1839426864 tempest-VolumesNegativeTest-1839426864-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:28:24.189250 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-e84240a7-30a7-485e-8cce-0e248c8c2409 tempest-VolumesNegativeTest-1839426864 tempest-VolumesNegativeTest-1839426864-project] [instance: ac07d059-1555-4f61-9a85-28396691a934] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:28:24.192255 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-36681c3a-a271-4c76-88b6-03a2fa86930a tempest-VolumesSnapshotTestJSON-1750456804 tempest-VolumesSnapshotTestJSON-1750456804-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:28:24.194092 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-36681c3a-a271-4c76-88b6-03a2fa86930a tempest-VolumesSnapshotTestJSON-1750456804 tempest-VolumesSnapshotTestJSON-1750456804-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:28:24.194092 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-36681c3a-a271-4c76-88b6-03a2fa86930a tempest-VolumesSnapshotTestJSON-1750456804 tempest-VolumesSnapshotTestJSON-1750456804-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:28:24.194092 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-36681c3a-a271-4c76-88b6-03a2fa86930a tempest-VolumesSnapshotTestJSON-1750456804 tempest-VolumesSnapshotTestJSON-1750456804-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:28:24.194477 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-36681c3a-a271-4c76-88b6-03a2fa86930a tempest-VolumesSnapshotTestJSON-1750456804 tempest-VolumesSnapshotTestJSON-1750456804-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:28:24.194834 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-36681c3a-a271-4c76-88b6-03a2fa86930a tempest-VolumesSnapshotTestJSON-1750456804 tempest-VolumesSnapshotTestJSON-1750456804-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:28:24.196196 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-e84240a7-30a7-485e-8cce-0e248c8c2409 tempest-VolumesNegativeTest-1839426864 tempest-VolumesNegativeTest-1839426864-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:28:24.196771 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-e84240a7-30a7-485e-8cce-0e248c8c2409 tempest-VolumesNegativeTest-1839426864 tempest-VolumesNegativeTest-1839426864-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:28:24.197180 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-e84240a7-30a7-485e-8cce-0e248c8c2409 tempest-VolumesNegativeTest-1839426864 tempest-VolumesNegativeTest-1839426864-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:28:24.198169 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-e84240a7-30a7-485e-8cce-0e248c8c2409 tempest-VolumesNegativeTest-1839426864 tempest-VolumesNegativeTest-1839426864-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:28:24.198613 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-e84240a7-30a7-485e-8cce-0e248c8c2409 tempest-VolumesNegativeTest-1839426864 tempest-VolumesNegativeTest-1839426864-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:28:24.200435 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-e84240a7-30a7-485e-8cce-0e248c8c2409 tempest-VolumesNegativeTest-1839426864 tempest-VolumesNegativeTest-1839426864-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:28:24.406325 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-36681c3a-a271-4c76-88b6-03a2fa86930a tempest-VolumesSnapshotTestJSON-1750456804 tempest-VolumesSnapshotTestJSON-1750456804-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '75744eccc3d24d4c9282cbe3a3314489', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fa43d54225bf44b283f9db5ddbbb13ca', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:28:24.410381 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-e84240a7-30a7-485e-8cce-0e248c8c2409 tempest-VolumesNegativeTest-1839426864 tempest-VolumesNegativeTest-1839426864-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '544e6c056b854bfdb9e117be686635a0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d1bead2e4f4102a53f582a4815661d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:28:24.414508 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-36681c3a-a271-4c76-88b6-03a2fa86930a tempest-VolumesSnapshotTestJSON-1750456804 tempest-VolumesSnapshotTestJSON-1750456804-project] 10.222.0.52 "GET /compute/v2.1/servers/46dae88e-3429-47c7-9376-c4c0d9cddd29" status: 200 len: 1513 microversion: 2.1 time: 0.233234 Aug 27 02:28:24.415093 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 309/616] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:28:24 2022] GET /compute/v2.1/servers/46dae88e-3429-47c7-9376-c4c0d9cddd29 => generated 1513 bytes in 234 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:28:24.425310 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-e84240a7-30a7-485e-8cce-0e248c8c2409 tempest-VolumesNegativeTest-1839426864 tempest-VolumesNegativeTest-1839426864-project] 10.222.0.52 "GET /compute/v2.1/servers/ac07d059-1555-4f61-9a85-28396691a934" status: 200 len: 1314 microversion: 2.1 time: 0.234525 Aug 27 02:28:24.426903 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 308/617] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:28:24 2022] GET /compute/v2.1/servers/ac07d059-1555-4f61-9a85-28396691a934 => generated 1314 bytes in 244 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 27 02:28:25.128650 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-4376e174-aa50-4c3a-812b-d29812428093 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Calling method '>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:28:25.129293 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-4376e174-aa50-4c3a-812b-d29812428093 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] [instance: 56dbfa6a-5b6f-4b82-977b-af735e6a6ac9] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:28:25.136111 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-4376e174-aa50-4c3a-812b-d29812428093 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:28:25.136592 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-4376e174-aa50-4c3a-812b-d29812428093 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:28:25.137018 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-4376e174-aa50-4c3a-812b-d29812428093 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:28:25.137742 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-4376e174-aa50-4c3a-812b-d29812428093 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:28:25.138185 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-4376e174-aa50-4c3a-812b-d29812428093 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:28:25.138603 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-4376e174-aa50-4c3a-812b-d29812428093 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:28:25.283816 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-4376e174-aa50-4c3a-812b-d29812428093 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:28:25.294985 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-4376e174-aa50-4c3a-812b-d29812428093 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:28:25.297550 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-4376e174-aa50-4c3a-812b-d29812428093 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:28:25.299566 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-4376e174-aa50-4c3a-812b-d29812428093 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] 10.222.0.52 "GET /compute/v2.1/servers/56dbfa6a-5b6f-4b82-977b-af735e6a6ac9" status: 200 len: 1724 microversion: 2.20 time: 0.174661 Aug 27 02:28:25.300511 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 310/618] 10.222.0.52 () {60 vars in 1277 bytes} [Sat Aug 27 02:28:25 2022] GET /compute/v2.1/servers/56dbfa6a-5b6f-4b82-977b-af735e6a6ac9 => generated 1724 bytes in 176 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 27 02:28:25.441048 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-78f53cdb-46e0-469a-937a-ed54072bfc9e tempest-VolumesSnapshotTestJSON-1750456804 tempest-VolumesSnapshotTestJSON-1750456804-project] Calling method '>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:28:25.441332 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-a60f0593-7076-421b-991e-539827763e33 tempest-VolumesNegativeTest-1839426864 tempest-VolumesNegativeTest-1839426864-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:28:25.441763 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-78f53cdb-46e0-469a-937a-ed54072bfc9e tempest-VolumesSnapshotTestJSON-1750456804 tempest-VolumesSnapshotTestJSON-1750456804-project] [instance: 46dae88e-3429-47c7-9376-c4c0d9cddd29] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:28:25.441847 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-a60f0593-7076-421b-991e-539827763e33 tempest-VolumesNegativeTest-1839426864 tempest-VolumesNegativeTest-1839426864-project] [instance: ac07d059-1555-4f61-9a85-28396691a934] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:28:25.448578 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-78f53cdb-46e0-469a-937a-ed54072bfc9e tempest-VolumesSnapshotTestJSON-1750456804 tempest-VolumesSnapshotTestJSON-1750456804-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:28:25.448792 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-a60f0593-7076-421b-991e-539827763e33 tempest-VolumesNegativeTest-1839426864 tempest-VolumesNegativeTest-1839426864-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:28:25.449060 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-78f53cdb-46e0-469a-937a-ed54072bfc9e tempest-VolumesSnapshotTestJSON-1750456804 tempest-VolumesSnapshotTestJSON-1750456804-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:28:25.449185 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-a60f0593-7076-421b-991e-539827763e33 tempest-VolumesNegativeTest-1839426864 tempest-VolumesNegativeTest-1839426864-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:28:25.449527 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-a60f0593-7076-421b-991e-539827763e33 tempest-VolumesNegativeTest-1839426864 tempest-VolumesNegativeTest-1839426864-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:28:25.449635 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-78f53cdb-46e0-469a-937a-ed54072bfc9e tempest-VolumesSnapshotTestJSON-1750456804 tempest-VolumesSnapshotTestJSON-1750456804-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:28:25.450262 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-78f53cdb-46e0-469a-937a-ed54072bfc9e tempest-VolumesSnapshotTestJSON-1750456804 tempest-VolumesSnapshotTestJSON-1750456804-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:28:25.450377 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-a60f0593-7076-421b-991e-539827763e33 tempest-VolumesNegativeTest-1839426864 tempest-VolumesNegativeTest-1839426864-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:28:25.450701 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-78f53cdb-46e0-469a-937a-ed54072bfc9e tempest-VolumesSnapshotTestJSON-1750456804 tempest-VolumesSnapshotTestJSON-1750456804-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:28:25.450800 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-a60f0593-7076-421b-991e-539827763e33 tempest-VolumesNegativeTest-1839426864 tempest-VolumesNegativeTest-1839426864-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:28:25.451119 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-a60f0593-7076-421b-991e-539827763e33 tempest-VolumesNegativeTest-1839426864 tempest-VolumesNegativeTest-1839426864-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:28:25.451214 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-78f53cdb-46e0-469a-937a-ed54072bfc9e tempest-VolumesSnapshotTestJSON-1750456804 tempest-VolumesSnapshotTestJSON-1750456804-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:28:25.648591 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-78f53cdb-46e0-469a-937a-ed54072bfc9e tempest-VolumesSnapshotTestJSON-1750456804 tempest-VolumesSnapshotTestJSON-1750456804-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '75744eccc3d24d4c9282cbe3a3314489', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fa43d54225bf44b283f9db5ddbbb13ca', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:28:25.650900 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-a60f0593-7076-421b-991e-539827763e33 tempest-VolumesNegativeTest-1839426864 tempest-VolumesNegativeTest-1839426864-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '544e6c056b854bfdb9e117be686635a0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d1bead2e4f4102a53f582a4815661d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:28:25.660186 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-78f53cdb-46e0-469a-937a-ed54072bfc9e tempest-VolumesSnapshotTestJSON-1750456804 tempest-VolumesSnapshotTestJSON-1750456804-project] 10.222.0.52 "GET /compute/v2.1/servers/46dae88e-3429-47c7-9376-c4c0d9cddd29" status: 200 len: 1513 microversion: 2.1 time: 0.220613 Aug 27 02:28:25.660186 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 311/619] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:28:25 2022] GET /compute/v2.1/servers/46dae88e-3429-47c7-9376-c4c0d9cddd29 => generated 1513 bytes in 222 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:28:25.661075 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-a60f0593-7076-421b-991e-539827763e33 tempest-VolumesNegativeTest-1839426864 tempest-VolumesNegativeTest-1839426864-project] 10.222.0.52 "GET /compute/v2.1/servers/ac07d059-1555-4f61-9a85-28396691a934" status: 200 len: 1314 microversion: 2.1 time: 0.224512 Aug 27 02:28:25.661671 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 309/620] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:28:25 2022] GET /compute/v2.1/servers/ac07d059-1555-4f61-9a85-28396691a934 => generated 1314 bytes in 226 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:28:26.327090 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-6f4d0aff-bd40-4312-bda8-7e9ab1ee910d tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Calling method '>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:28:26.327656 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-6f4d0aff-bd40-4312-bda8-7e9ab1ee910d tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] [instance: 56dbfa6a-5b6f-4b82-977b-af735e6a6ac9] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:28:26.334557 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-6f4d0aff-bd40-4312-bda8-7e9ab1ee910d tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:28:26.335008 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-6f4d0aff-bd40-4312-bda8-7e9ab1ee910d tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:28:26.335433 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-6f4d0aff-bd40-4312-bda8-7e9ab1ee910d tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:28:26.336150 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-6f4d0aff-bd40-4312-bda8-7e9ab1ee910d tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:28:26.336568 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-6f4d0aff-bd40-4312-bda8-7e9ab1ee910d tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:28:26.336988 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-6f4d0aff-bd40-4312-bda8-7e9ab1ee910d tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:28:26.479001 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-6f4d0aff-bd40-4312-bda8-7e9ab1ee910d tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:28:26.481329 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG dbcounter [-] [120864] Writing DB stats nova_cell1:SELECT=97,nova_cell1:UPDATE=2,nova_cell1:INSERT=1 {{(pid=120864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 27 02:28:26.488509 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-6f4d0aff-bd40-4312-bda8-7e9ab1ee910d tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:28:26.490061 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-6f4d0aff-bd40-4312-bda8-7e9ab1ee910d tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:28:26.491251 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-6f4d0aff-bd40-4312-bda8-7e9ab1ee910d tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] 10.222.0.52 "GET /compute/v2.1/servers/56dbfa6a-5b6f-4b82-977b-af735e6a6ac9" status: 200 len: 1724 microversion: 2.20 time: 0.168140 Aug 27 02:28:26.491807 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 312/621] 10.222.0.52 () {60 vars in 1277 bytes} [Sat Aug 27 02:28:26 2022] GET /compute/v2.1/servers/56dbfa6a-5b6f-4b82-977b-af735e6a6ac9 => generated 1724 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 27 02:28:26.680120 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-760f1333-a260-434b-a457-ea84d2051ad0 tempest-VolumesSnapshotTestJSON-1750456804 tempest-VolumesSnapshotTestJSON-1750456804-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:28:26.681714 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-760f1333-a260-434b-a457-ea84d2051ad0 tempest-VolumesSnapshotTestJSON-1750456804 tempest-VolumesSnapshotTestJSON-1750456804-project] [instance: 46dae88e-3429-47c7-9376-c4c0d9cddd29] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:28:26.684858 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-6d272f6e-94e0-4cbf-b135-149e1da44845 tempest-VolumesNegativeTest-1839426864 tempest-VolumesNegativeTest-1839426864-project] Calling method '>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:28:26.685488 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-6d272f6e-94e0-4cbf-b135-149e1da44845 tempest-VolumesNegativeTest-1839426864 tempest-VolumesNegativeTest-1839426864-project] [instance: ac07d059-1555-4f61-9a85-28396691a934] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:28:26.692833 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-6d272f6e-94e0-4cbf-b135-149e1da44845 tempest-VolumesNegativeTest-1839426864 tempest-VolumesNegativeTest-1839426864-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:28:26.693491 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-6d272f6e-94e0-4cbf-b135-149e1da44845 tempest-VolumesNegativeTest-1839426864 tempest-VolumesNegativeTest-1839426864-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:28:26.694162 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-6d272f6e-94e0-4cbf-b135-149e1da44845 tempest-VolumesNegativeTest-1839426864 tempest-VolumesNegativeTest-1839426864-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:28:26.694564 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-6d272f6e-94e0-4cbf-b135-149e1da44845 tempest-VolumesNegativeTest-1839426864 tempest-VolumesNegativeTest-1839426864-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:28:26.695012 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-6d272f6e-94e0-4cbf-b135-149e1da44845 tempest-VolumesNegativeTest-1839426864 tempest-VolumesNegativeTest-1839426864-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:28:26.695428 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-6d272f6e-94e0-4cbf-b135-149e1da44845 tempest-VolumesNegativeTest-1839426864 tempest-VolumesNegativeTest-1839426864-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:28:26.697845 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-760f1333-a260-434b-a457-ea84d2051ad0 tempest-VolumesSnapshotTestJSON-1750456804 tempest-VolumesSnapshotTestJSON-1750456804-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:28:26.698350 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-760f1333-a260-434b-a457-ea84d2051ad0 tempest-VolumesSnapshotTestJSON-1750456804 tempest-VolumesSnapshotTestJSON-1750456804-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:28:26.698788 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-760f1333-a260-434b-a457-ea84d2051ad0 tempest-VolumesSnapshotTestJSON-1750456804 tempest-VolumesSnapshotTestJSON-1750456804-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:28:26.699492 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-760f1333-a260-434b-a457-ea84d2051ad0 tempest-VolumesSnapshotTestJSON-1750456804 tempest-VolumesSnapshotTestJSON-1750456804-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:28:26.699941 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-760f1333-a260-434b-a457-ea84d2051ad0 tempest-VolumesSnapshotTestJSON-1750456804 tempest-VolumesSnapshotTestJSON-1750456804-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:28:26.700353 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-760f1333-a260-434b-a457-ea84d2051ad0 tempest-VolumesSnapshotTestJSON-1750456804 tempest-VolumesSnapshotTestJSON-1750456804-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:28:26.899586 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-6d272f6e-94e0-4cbf-b135-149e1da44845 tempest-VolumesNegativeTest-1839426864 tempest-VolumesNegativeTest-1839426864-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '544e6c056b854bfdb9e117be686635a0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d1bead2e4f4102a53f582a4815661d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:28:26.903126 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-760f1333-a260-434b-a457-ea84d2051ad0 tempest-VolumesSnapshotTestJSON-1750456804 tempest-VolumesSnapshotTestJSON-1750456804-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '75744eccc3d24d4c9282cbe3a3314489', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fa43d54225bf44b283f9db5ddbbb13ca', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:28:26.910393 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-6d272f6e-94e0-4cbf-b135-149e1da44845 tempest-VolumesNegativeTest-1839426864 tempest-VolumesNegativeTest-1839426864-project] 10.222.0.52 "GET /compute/v2.1/servers/ac07d059-1555-4f61-9a85-28396691a934" status: 200 len: 1314 microversion: 2.1 time: 0.231853 Aug 27 02:28:26.911056 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 313/622] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:28:26 2022] GET /compute/v2.1/servers/ac07d059-1555-4f61-9a85-28396691a934 => generated 1314 bytes in 233 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 27 02:28:26.911633 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-760f1333-a260-434b-a457-ea84d2051ad0 tempest-VolumesSnapshotTestJSON-1750456804 tempest-VolumesSnapshotTestJSON-1750456804-project] 10.222.0.52 "GET /compute/v2.1/servers/46dae88e-3429-47c7-9376-c4c0d9cddd29" status: 200 len: 1513 microversion: 2.1 time: 0.235821 Aug 27 02:28:26.912319 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 310/623] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:28:26 2022] GET /compute/v2.1/servers/46dae88e-3429-47c7-9376-c4c0d9cddd29 => generated 1513 bytes in 237 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:28:27.517408 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-0529997f-63b5-416d-bdcd-5de69ad3a082 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Calling method '>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:28:27.518507 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-0529997f-63b5-416d-bdcd-5de69ad3a082 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] [instance: 56dbfa6a-5b6f-4b82-977b-af735e6a6ac9] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:28:27.529797 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-0529997f-63b5-416d-bdcd-5de69ad3a082 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:28:27.530484 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-0529997f-63b5-416d-bdcd-5de69ad3a082 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:28:27.530985 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-0529997f-63b5-416d-bdcd-5de69ad3a082 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:28:27.532383 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-0529997f-63b5-416d-bdcd-5de69ad3a082 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:28:27.532961 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-0529997f-63b5-416d-bdcd-5de69ad3a082 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:28:27.533995 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-0529997f-63b5-416d-bdcd-5de69ad3a082 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:28:27.677948 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-0529997f-63b5-416d-bdcd-5de69ad3a082 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:28:27.686655 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-0529997f-63b5-416d-bdcd-5de69ad3a082 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:28:27.688243 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-0529997f-63b5-416d-bdcd-5de69ad3a082 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:28:27.689451 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-0529997f-63b5-416d-bdcd-5de69ad3a082 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] 10.222.0.52 "GET /compute/v2.1/servers/56dbfa6a-5b6f-4b82-977b-af735e6a6ac9" status: 200 len: 1724 microversion: 2.20 time: 0.178509 Aug 27 02:28:27.690010 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 314/624] 10.222.0.52 () {60 vars in 1277 bytes} [Sat Aug 27 02:28:27 2022] GET /compute/v2.1/servers/56dbfa6a-5b6f-4b82-977b-af735e6a6ac9 => generated 1724 bytes in 180 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 27 02:28:27.937297 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-786f2c6e-0a7f-4ebc-ab06-1f204df0aedc tempest-VolumesSnapshotTestJSON-1750456804 tempest-VolumesSnapshotTestJSON-1750456804-project] Calling method '>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:28:27.938270 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-786f2c6e-0a7f-4ebc-ab06-1f204df0aedc tempest-VolumesSnapshotTestJSON-1750456804 tempest-VolumesSnapshotTestJSON-1750456804-project] [instance: 46dae88e-3429-47c7-9376-c4c0d9cddd29] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:28:27.940541 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-72d8c905-a898-4899-9ea9-c692734090b8 tempest-VolumesNegativeTest-1839426864 tempest-VolumesNegativeTest-1839426864-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:28:27.941181 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-72d8c905-a898-4899-9ea9-c692734090b8 tempest-VolumesNegativeTest-1839426864 tempest-VolumesNegativeTest-1839426864-project] [instance: ac07d059-1555-4f61-9a85-28396691a934] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:28:27.944599 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-786f2c6e-0a7f-4ebc-ab06-1f204df0aedc tempest-VolumesSnapshotTestJSON-1750456804 tempest-VolumesSnapshotTestJSON-1750456804-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:28:27.945089 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-786f2c6e-0a7f-4ebc-ab06-1f204df0aedc tempest-VolumesSnapshotTestJSON-1750456804 tempest-VolumesSnapshotTestJSON-1750456804-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:28:27.945541 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-786f2c6e-0a7f-4ebc-ab06-1f204df0aedc tempest-VolumesSnapshotTestJSON-1750456804 tempest-VolumesSnapshotTestJSON-1750456804-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:28:27.946251 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-786f2c6e-0a7f-4ebc-ab06-1f204df0aedc tempest-VolumesSnapshotTestJSON-1750456804 tempest-VolumesSnapshotTestJSON-1750456804-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:28:27.946686 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-786f2c6e-0a7f-4ebc-ab06-1f204df0aedc tempest-VolumesSnapshotTestJSON-1750456804 tempest-VolumesSnapshotTestJSON-1750456804-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:28:27.947099 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-786f2c6e-0a7f-4ebc-ab06-1f204df0aedc tempest-VolumesSnapshotTestJSON-1750456804 tempest-VolumesSnapshotTestJSON-1750456804-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:28:27.948647 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-72d8c905-a898-4899-9ea9-c692734090b8 tempest-VolumesNegativeTest-1839426864 tempest-VolumesNegativeTest-1839426864-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:28:27.949167 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-72d8c905-a898-4899-9ea9-c692734090b8 tempest-VolumesNegativeTest-1839426864 tempest-VolumesNegativeTest-1839426864-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:28:27.949606 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-72d8c905-a898-4899-9ea9-c692734090b8 tempest-VolumesNegativeTest-1839426864 tempest-VolumesNegativeTest-1839426864-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:28:27.950527 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-72d8c905-a898-4899-9ea9-c692734090b8 tempest-VolumesNegativeTest-1839426864 tempest-VolumesNegativeTest-1839426864-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:28:27.950981 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-72d8c905-a898-4899-9ea9-c692734090b8 tempest-VolumesNegativeTest-1839426864 tempest-VolumesNegativeTest-1839426864-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:28:27.951570 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-72d8c905-a898-4899-9ea9-c692734090b8 tempest-VolumesNegativeTest-1839426864 tempest-VolumesNegativeTest-1839426864-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:28:27.954585 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG dbcounter [-] [120863] Writing DB stats nova_cell1:SELECT=97,nova_cell1:INSERT=2,nova_cell1:UPDATE=1 {{(pid=120863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 27 02:28:28.162206 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-786f2c6e-0a7f-4ebc-ab06-1f204df0aedc tempest-VolumesSnapshotTestJSON-1750456804 tempest-VolumesSnapshotTestJSON-1750456804-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '75744eccc3d24d4c9282cbe3a3314489', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fa43d54225bf44b283f9db5ddbbb13ca', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:28:28.171030 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-786f2c6e-0a7f-4ebc-ab06-1f204df0aedc tempest-VolumesSnapshotTestJSON-1750456804 tempest-VolumesSnapshotTestJSON-1750456804-project] 10.222.0.52 "GET /compute/v2.1/servers/46dae88e-3429-47c7-9376-c4c0d9cddd29" status: 200 len: 1513 microversion: 2.1 time: 0.237613 Aug 27 02:28:28.172156 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 315/625] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:28:27 2022] GET /compute/v2.1/servers/46dae88e-3429-47c7-9376-c4c0d9cddd29 => generated 1513 bytes in 239 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:28:28.184496 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-72d8c905-a898-4899-9ea9-c692734090b8 tempest-VolumesNegativeTest-1839426864 tempest-VolumesNegativeTest-1839426864-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '544e6c056b854bfdb9e117be686635a0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d1bead2e4f4102a53f582a4815661d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:28:28.192445 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-72d8c905-a898-4899-9ea9-c692734090b8 tempest-VolumesNegativeTest-1839426864 tempest-VolumesNegativeTest-1839426864-project] 10.222.0.52 "GET /compute/v2.1/servers/ac07d059-1555-4f61-9a85-28396691a934" status: 200 len: 1464 microversion: 2.1 time: 0.262886 Aug 27 02:28:28.193034 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 311/626] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:28:27 2022] GET /compute/v2.1/servers/ac07d059-1555-4f61-9a85-28396691a934 => generated 1464 bytes in 265 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:28:28.232534 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [req-4886ffcc-15ed-4eb7-ad38-2fe0cb9c5ad4 req-a6d10fea-f159-4cab-afed-de0b87769332 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "46dae88e-3429-47c7-9376-c4c0d9cddd29", "name": "network-vif-deleted", "tag": "692449b2-a058-408c-bf04-f924a65e0f5a"}]} {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 27 02:28:28.246450 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-e002037f-254a-4f11-808f-ca848b7164d9 tempest-VolumesNegativeTest-1839426864 tempest-VolumesNegativeTest-1839426864-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:28:28.247041 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-e002037f-254a-4f11-808f-ca848b7164d9 tempest-VolumesNegativeTest-1839426864 tempest-VolumesNegativeTest-1839426864-project] [instance: ac07d059-1555-4f61-9a85-28396691a934] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:28:28.252739 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [req-4886ffcc-15ed-4eb7-ad38-2fe0cb9c5ad4 req-a6d10fea-f159-4cab-afed-de0b87769332 service nova] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:28:28.253155 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [req-4886ffcc-15ed-4eb7-ad38-2fe0cb9c5ad4 req-a6d10fea-f159-4cab-afed-de0b87769332 service nova] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:28:28.253779 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [req-4886ffcc-15ed-4eb7-ad38-2fe0cb9c5ad4 req-a6d10fea-f159-4cab-afed-de0b87769332 service nova] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:28:28.254234 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-e002037f-254a-4f11-808f-ca848b7164d9 tempest-VolumesNegativeTest-1839426864 tempest-VolumesNegativeTest-1839426864-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:28:28.254907 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-e002037f-254a-4f11-808f-ca848b7164d9 tempest-VolumesNegativeTest-1839426864 tempest-VolumesNegativeTest-1839426864-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:28:28.256794 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-e002037f-254a-4f11-808f-ca848b7164d9 tempest-VolumesNegativeTest-1839426864 tempest-VolumesNegativeTest-1839426864-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:28:28.258090 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-e002037f-254a-4f11-808f-ca848b7164d9 tempest-VolumesNegativeTest-1839426864 tempest-VolumesNegativeTest-1839426864-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:28:28.258678 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-e002037f-254a-4f11-808f-ca848b7164d9 tempest-VolumesNegativeTest-1839426864 tempest-VolumesNegativeTest-1839426864-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:28:28.259126 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-e002037f-254a-4f11-808f-ca848b7164d9 tempest-VolumesNegativeTest-1839426864 tempest-VolumesNegativeTest-1839426864-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:28:28.268182 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.compute.server_external_events [req-4886ffcc-15ed-4eb7-ad38-2fe0cb9c5ad4 req-a6d10fea-f159-4cab-afed-de0b87769332 service nova] Creating event network-vif-deleted:692449b2-a058-408c-bf04-f924a65e0f5a for instance 46dae88e-3429-47c7-9376-c4c0d9cddd29 on hv-cinder-84630 Aug 27 02:28:28.276233 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [req-4886ffcc-15ed-4eb7-ad38-2fe0cb9c5ad4 req-a6d10fea-f159-4cab-afed-de0b87769332 service nova] 10.222.0.52 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.047215 Aug 27 02:28:28.277286 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 316/627] 10.222.0.52 () {64 vars in 1317 bytes} [Sat Aug 27 02:28:28 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 27 02:28:28.308369 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-e002037f-254a-4f11-808f-ca848b7164d9 tempest-VolumesNegativeTest-1839426864 tempest-VolumesNegativeTest-1839426864-project] [instance: ac07d059-1555-4f61-9a85-28396691a934] Going to try to terminate instance {{(pid=120863) delete /opt/stack/nova/nova/compute/api.py:2719}} Aug 27 02:28:28.398212 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-e002037f-254a-4f11-808f-ca848b7164d9 tempest-VolumesNegativeTest-1839426864 tempest-VolumesNegativeTest-1839426864-project] 10.222.0.52 "DELETE /compute/v2.1/servers/ac07d059-1555-4f61-9a85-28396691a934" status: 204 len: 0 microversion: 2.1 time: 0.155173 Aug 27 02:28:28.398728 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 312/628] 10.222.0.52 () {58 vars in 1239 bytes} [Sat Aug 27 02:28:28 2022] DELETE /compute/v2.1/servers/ac07d059-1555-4f61-9a85-28396691a934 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 9 headers in 363 bytes (2 switches on core 0) Aug 27 02:28:28.408183 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-06193cfa-f259-4b06-8041-f0f64178ccbb tempest-VolumesNegativeTest-1839426864 tempest-VolumesNegativeTest-1839426864-project] Calling method '>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:28:28.408846 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-06193cfa-f259-4b06-8041-f0f64178ccbb tempest-VolumesNegativeTest-1839426864 tempest-VolumesNegativeTest-1839426864-project] [instance: ac07d059-1555-4f61-9a85-28396691a934] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:28:28.415499 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-06193cfa-f259-4b06-8041-f0f64178ccbb tempest-VolumesNegativeTest-1839426864 tempest-VolumesNegativeTest-1839426864-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:28:28.415990 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-06193cfa-f259-4b06-8041-f0f64178ccbb tempest-VolumesNegativeTest-1839426864 tempest-VolumesNegativeTest-1839426864-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:28:28.416413 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-06193cfa-f259-4b06-8041-f0f64178ccbb tempest-VolumesNegativeTest-1839426864 tempest-VolumesNegativeTest-1839426864-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:28:28.417097 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-06193cfa-f259-4b06-8041-f0f64178ccbb tempest-VolumesNegativeTest-1839426864 tempest-VolumesNegativeTest-1839426864-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:28:28.417671 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-06193cfa-f259-4b06-8041-f0f64178ccbb tempest-VolumesNegativeTest-1839426864 tempest-VolumesNegativeTest-1839426864-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:28:28.418160 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-06193cfa-f259-4b06-8041-f0f64178ccbb tempest-VolumesNegativeTest-1839426864 tempest-VolumesNegativeTest-1839426864-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:28:28.556916 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-06193cfa-f259-4b06-8041-f0f64178ccbb tempest-VolumesNegativeTest-1839426864 tempest-VolumesNegativeTest-1839426864-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '544e6c056b854bfdb9e117be686635a0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d1bead2e4f4102a53f582a4815661d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:28:28.564928 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-06193cfa-f259-4b06-8041-f0f64178ccbb tempest-VolumesNegativeTest-1839426864 tempest-VolumesNegativeTest-1839426864-project] 10.222.0.52 "GET /compute/v2.1/servers/ac07d059-1555-4f61-9a85-28396691a934" status: 200 len: 1470 microversion: 2.1 time: 0.161391 Aug 27 02:28:28.565520 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 317/629] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:28:28 2022] GET /compute/v2.1/servers/ac07d059-1555-4f61-9a85-28396691a934 => generated 1470 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 27 02:28:28.716030 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-0b9073ef-afdd-4540-9464-96a9f3fdfaf6 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:28:28.717236 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-0b9073ef-afdd-4540-9464-96a9f3fdfaf6 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] [instance: 56dbfa6a-5b6f-4b82-977b-af735e6a6ac9] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:28:28.725541 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-0b9073ef-afdd-4540-9464-96a9f3fdfaf6 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:28:28.726388 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-0b9073ef-afdd-4540-9464-96a9f3fdfaf6 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:28:28.727110 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-0b9073ef-afdd-4540-9464-96a9f3fdfaf6 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:28:28.728238 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-0b9073ef-afdd-4540-9464-96a9f3fdfaf6 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:28:28.728933 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-0b9073ef-afdd-4540-9464-96a9f3fdfaf6 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:28:28.729708 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-0b9073ef-afdd-4540-9464-96a9f3fdfaf6 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:28:28.873685 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-0b9073ef-afdd-4540-9464-96a9f3fdfaf6 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:28:28.883555 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-0b9073ef-afdd-4540-9464-96a9f3fdfaf6 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:28:28.885005 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-0b9073ef-afdd-4540-9464-96a9f3fdfaf6 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:28:28.886320 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-0b9073ef-afdd-4540-9464-96a9f3fdfaf6 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] 10.222.0.52 "GET /compute/v2.1/servers/56dbfa6a-5b6f-4b82-977b-af735e6a6ac9" status: 200 len: 1724 microversion: 2.20 time: 0.176548 Aug 27 02:28:28.886959 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 313/630] 10.222.0.52 () {60 vars in 1277 bytes} [Sat Aug 27 02:28:28 2022] GET /compute/v2.1/servers/56dbfa6a-5b6f-4b82-977b-af735e6a6ac9 => generated 1724 bytes in 178 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 27 02:28:29.194187 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-e126ec02-8bde-4ec8-814f-8db4af62f660 tempest-VolumesSnapshotTestJSON-1750456804 tempest-VolumesSnapshotTestJSON-1750456804-project] Calling method '>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:28:29.195098 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-e126ec02-8bde-4ec8-814f-8db4af62f660 tempest-VolumesSnapshotTestJSON-1750456804 tempest-VolumesSnapshotTestJSON-1750456804-project] [instance: 46dae88e-3429-47c7-9376-c4c0d9cddd29] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:28:29.205052 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-e126ec02-8bde-4ec8-814f-8db4af62f660 tempest-VolumesSnapshotTestJSON-1750456804 tempest-VolumesSnapshotTestJSON-1750456804-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:28:29.205815 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-e126ec02-8bde-4ec8-814f-8db4af62f660 tempest-VolumesSnapshotTestJSON-1750456804 tempest-VolumesSnapshotTestJSON-1750456804-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:28:29.206728 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-e126ec02-8bde-4ec8-814f-8db4af62f660 tempest-VolumesSnapshotTestJSON-1750456804 tempest-VolumesSnapshotTestJSON-1750456804-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:28:29.207776 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-e126ec02-8bde-4ec8-814f-8db4af62f660 tempest-VolumesSnapshotTestJSON-1750456804 tempest-VolumesSnapshotTestJSON-1750456804-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:28:29.208315 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-e126ec02-8bde-4ec8-814f-8db4af62f660 tempest-VolumesSnapshotTestJSON-1750456804 tempest-VolumesSnapshotTestJSON-1750456804-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:28:29.208944 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-e126ec02-8bde-4ec8-814f-8db4af62f660 tempest-VolumesSnapshotTestJSON-1750456804 tempest-VolumesSnapshotTestJSON-1750456804-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:28:29.247795 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-e126ec02-8bde-4ec8-814f-8db4af62f660 tempest-VolumesSnapshotTestJSON-1750456804 tempest-VolumesSnapshotTestJSON-1750456804-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:28:29.248310 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-e126ec02-8bde-4ec8-814f-8db4af62f660 tempest-VolumesSnapshotTestJSON-1750456804 tempest-VolumesSnapshotTestJSON-1750456804-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:28:29.248850 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-e126ec02-8bde-4ec8-814f-8db4af62f660 tempest-VolumesSnapshotTestJSON-1750456804 tempest-VolumesSnapshotTestJSON-1750456804-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:28:29.250220 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.objects.instance [None req-e126ec02-8bde-4ec8-814f-8db4af62f660 tempest-VolumesSnapshotTestJSON-1750456804 tempest-VolumesSnapshotTestJSON-1750456804-project] Lazy-loading 'fault' on Instance uuid 46dae88e-3429-47c7-9376-c4c0d9cddd29 {{(pid=120864) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Aug 27 02:28:29.293169 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-e126ec02-8bde-4ec8-814f-8db4af62f660 tempest-VolumesSnapshotTestJSON-1750456804 tempest-VolumesSnapshotTestJSON-1750456804-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '75744eccc3d24d4c9282cbe3a3314489', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fa43d54225bf44b283f9db5ddbbb13ca', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:28:29.301358 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-e126ec02-8bde-4ec8-814f-8db4af62f660 tempest-VolumesSnapshotTestJSON-1750456804 tempest-VolumesSnapshotTestJSON-1750456804-project] 10.222.0.52 "GET /compute/v2.1/servers/46dae88e-3429-47c7-9376-c4c0d9cddd29" status: 200 len: 1308 microversion: 2.1 time: 0.113086 Aug 27 02:28:29.301937 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 318/631] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:28:29 2022] GET /compute/v2.1/servers/46dae88e-3429-47c7-9376-c4c0d9cddd29 => generated 1308 bytes in 114 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:28:29.591562 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-befaba5a-f6dd-4be2-8872-7e469ee685e3 tempest-VolumesNegativeTest-1839426864 tempest-VolumesNegativeTest-1839426864-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:28:29.592543 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-befaba5a-f6dd-4be2-8872-7e469ee685e3 tempest-VolumesNegativeTest-1839426864 tempest-VolumesNegativeTest-1839426864-project] [instance: ac07d059-1555-4f61-9a85-28396691a934] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:28:29.602483 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-befaba5a-f6dd-4be2-8872-7e469ee685e3 tempest-VolumesNegativeTest-1839426864 tempest-VolumesNegativeTest-1839426864-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:28:29.603260 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-befaba5a-f6dd-4be2-8872-7e469ee685e3 tempest-VolumesNegativeTest-1839426864 tempest-VolumesNegativeTest-1839426864-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:28:29.603979 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-befaba5a-f6dd-4be2-8872-7e469ee685e3 tempest-VolumesNegativeTest-1839426864 tempest-VolumesNegativeTest-1839426864-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:28:29.605113 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-befaba5a-f6dd-4be2-8872-7e469ee685e3 tempest-VolumesNegativeTest-1839426864 tempest-VolumesNegativeTest-1839426864-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:28:29.605910 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-befaba5a-f6dd-4be2-8872-7e469ee685e3 tempest-VolumesNegativeTest-1839426864 tempest-VolumesNegativeTest-1839426864-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:28:29.606642 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-befaba5a-f6dd-4be2-8872-7e469ee685e3 tempest-VolumesNegativeTest-1839426864 tempest-VolumesNegativeTest-1839426864-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:28:29.757245 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-befaba5a-f6dd-4be2-8872-7e469ee685e3 tempest-VolumesNegativeTest-1839426864 tempest-VolumesNegativeTest-1839426864-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '544e6c056b854bfdb9e117be686635a0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d1bead2e4f4102a53f582a4815661d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:28:29.765769 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-befaba5a-f6dd-4be2-8872-7e469ee685e3 tempest-VolumesNegativeTest-1839426864 tempest-VolumesNegativeTest-1839426864-project] 10.222.0.52 "GET /compute/v2.1/servers/ac07d059-1555-4f61-9a85-28396691a934" status: 200 len: 1470 microversion: 2.1 time: 0.180456 Aug 27 02:28:29.766343 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 314/632] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:28:29 2022] GET /compute/v2.1/servers/ac07d059-1555-4f61-9a85-28396691a934 => generated 1470 bytes in 183 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:28:29.911781 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-a568f1d1-7709-4c50-b4bd-1c4abbacf505 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Calling method '>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:28:29.912998 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-a568f1d1-7709-4c50-b4bd-1c4abbacf505 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] [instance: 56dbfa6a-5b6f-4b82-977b-af735e6a6ac9] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:28:29.919982 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-a568f1d1-7709-4c50-b4bd-1c4abbacf505 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:28:29.920544 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-a568f1d1-7709-4c50-b4bd-1c4abbacf505 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:28:29.921080 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-a568f1d1-7709-4c50-b4bd-1c4abbacf505 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:28:29.921878 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-a568f1d1-7709-4c50-b4bd-1c4abbacf505 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:28:29.922326 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-a568f1d1-7709-4c50-b4bd-1c4abbacf505 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:28:29.922746 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-a568f1d1-7709-4c50-b4bd-1c4abbacf505 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:28:30.061267 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-a568f1d1-7709-4c50-b4bd-1c4abbacf505 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:28:30.069657 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-a568f1d1-7709-4c50-b4bd-1c4abbacf505 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:28:30.071111 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-a568f1d1-7709-4c50-b4bd-1c4abbacf505 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:28:30.072246 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-a568f1d1-7709-4c50-b4bd-1c4abbacf505 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] 10.222.0.52 "GET /compute/v2.1/servers/56dbfa6a-5b6f-4b82-977b-af735e6a6ac9" status: 200 len: 1724 microversion: 2.20 time: 0.164720 Aug 27 02:28:30.072799 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 319/633] 10.222.0.52 () {60 vars in 1277 bytes} [Sat Aug 27 02:28:29 2022] GET /compute/v2.1/servers/56dbfa6a-5b6f-4b82-977b-af735e6a6ac9 => generated 1724 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 27 02:28:30.323180 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-e5f84cdb-f803-42ba-bb1e-b7f5bc766ef7 tempest-VolumesSnapshotTestJSON-1750456804 tempest-VolumesSnapshotTestJSON-1750456804-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:28:30.323823 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-e5f84cdb-f803-42ba-bb1e-b7f5bc766ef7 tempest-VolumesSnapshotTestJSON-1750456804 tempest-VolumesSnapshotTestJSON-1750456804-project] [instance: 46dae88e-3429-47c7-9376-c4c0d9cddd29] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:28:30.330476 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-e5f84cdb-f803-42ba-bb1e-b7f5bc766ef7 tempest-VolumesSnapshotTestJSON-1750456804 tempest-VolumesSnapshotTestJSON-1750456804-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:28:30.331097 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-e5f84cdb-f803-42ba-bb1e-b7f5bc766ef7 tempest-VolumesSnapshotTestJSON-1750456804 tempest-VolumesSnapshotTestJSON-1750456804-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:28:30.331848 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-e5f84cdb-f803-42ba-bb1e-b7f5bc766ef7 tempest-VolumesSnapshotTestJSON-1750456804 tempest-VolumesSnapshotTestJSON-1750456804-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:28:30.332544 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-e5f84cdb-f803-42ba-bb1e-b7f5bc766ef7 tempest-VolumesSnapshotTestJSON-1750456804 tempest-VolumesSnapshotTestJSON-1750456804-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:28:30.333005 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-e5f84cdb-f803-42ba-bb1e-b7f5bc766ef7 tempest-VolumesSnapshotTestJSON-1750456804 tempest-VolumesSnapshotTestJSON-1750456804-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:28:30.333496 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-e5f84cdb-f803-42ba-bb1e-b7f5bc766ef7 tempest-VolumesSnapshotTestJSON-1750456804 tempest-VolumesSnapshotTestJSON-1750456804-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:28:30.371205 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-e5f84cdb-f803-42ba-bb1e-b7f5bc766ef7 tempest-VolumesSnapshotTestJSON-1750456804 tempest-VolumesSnapshotTestJSON-1750456804-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:28:30.371698 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-e5f84cdb-f803-42ba-bb1e-b7f5bc766ef7 tempest-VolumesSnapshotTestJSON-1750456804 tempest-VolumesSnapshotTestJSON-1750456804-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:28:30.372133 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-e5f84cdb-f803-42ba-bb1e-b7f5bc766ef7 tempest-VolumesSnapshotTestJSON-1750456804 tempest-VolumesSnapshotTestJSON-1750456804-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:28:30.372589 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.objects.instance [None req-e5f84cdb-f803-42ba-bb1e-b7f5bc766ef7 tempest-VolumesSnapshotTestJSON-1750456804 tempest-VolumesSnapshotTestJSON-1750456804-project] Lazy-loading 'fault' on Instance uuid 46dae88e-3429-47c7-9376-c4c0d9cddd29 {{(pid=120863) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Aug 27 02:28:30.420528 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-e5f84cdb-f803-42ba-bb1e-b7f5bc766ef7 tempest-VolumesSnapshotTestJSON-1750456804 tempest-VolumesSnapshotTestJSON-1750456804-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '75744eccc3d24d4c9282cbe3a3314489', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fa43d54225bf44b283f9db5ddbbb13ca', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:28:30.431273 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-e5f84cdb-f803-42ba-bb1e-b7f5bc766ef7 tempest-VolumesSnapshotTestJSON-1750456804 tempest-VolumesSnapshotTestJSON-1750456804-project] 10.222.0.52 "GET /compute/v2.1/servers/46dae88e-3429-47c7-9376-c4c0d9cddd29" status: 200 len: 1308 microversion: 2.1 time: 0.112169 Aug 27 02:28:30.431862 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 315/634] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:28:30 2022] GET /compute/v2.1/servers/46dae88e-3429-47c7-9376-c4c0d9cddd29 => generated 1308 bytes in 114 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:28:30.794059 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-1ed63f47-95ab-425f-8f5b-af68ad2f86a5 tempest-VolumesNegativeTest-1839426864 tempest-VolumesNegativeTest-1839426864-project] Calling method '>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:28:30.794812 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-1ed63f47-95ab-425f-8f5b-af68ad2f86a5 tempest-VolumesNegativeTest-1839426864 tempest-VolumesNegativeTest-1839426864-project] [instance: ac07d059-1555-4f61-9a85-28396691a934] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:28:30.801292 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-1ed63f47-95ab-425f-8f5b-af68ad2f86a5 tempest-VolumesNegativeTest-1839426864 tempest-VolumesNegativeTest-1839426864-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:28:30.801886 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-1ed63f47-95ab-425f-8f5b-af68ad2f86a5 tempest-VolumesNegativeTest-1839426864 tempest-VolumesNegativeTest-1839426864-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:28:30.802398 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-1ed63f47-95ab-425f-8f5b-af68ad2f86a5 tempest-VolumesNegativeTest-1839426864 tempest-VolumesNegativeTest-1839426864-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:28:30.803236 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-1ed63f47-95ab-425f-8f5b-af68ad2f86a5 tempest-VolumesNegativeTest-1839426864 tempest-VolumesNegativeTest-1839426864-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:28:30.803718 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-1ed63f47-95ab-425f-8f5b-af68ad2f86a5 tempest-VolumesNegativeTest-1839426864 tempest-VolumesNegativeTest-1839426864-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:28:30.804171 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-1ed63f47-95ab-425f-8f5b-af68ad2f86a5 tempest-VolumesNegativeTest-1839426864 tempest-VolumesNegativeTest-1839426864-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:28:30.982755 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-1ed63f47-95ab-425f-8f5b-af68ad2f86a5 tempest-VolumesNegativeTest-1839426864 tempest-VolumesNegativeTest-1839426864-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '544e6c056b854bfdb9e117be686635a0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d1bead2e4f4102a53f582a4815661d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:28:30.991731 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-1ed63f47-95ab-425f-8f5b-af68ad2f86a5 tempest-VolumesNegativeTest-1839426864 tempest-VolumesNegativeTest-1839426864-project] 10.222.0.52 "GET /compute/v2.1/servers/ac07d059-1555-4f61-9a85-28396691a934" status: 200 len: 1470 microversion: 2.1 time: 0.206378 Aug 27 02:28:30.992308 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 320/635] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:28:30 2022] GET /compute/v2.1/servers/ac07d059-1555-4f61-9a85-28396691a934 => generated 1470 bytes in 208 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:28:31.096282 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-b89c59bb-e482-47d8-b8c8-dd7fa327a59a tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:28:31.096993 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-b89c59bb-e482-47d8-b8c8-dd7fa327a59a tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] [instance: 56dbfa6a-5b6f-4b82-977b-af735e6a6ac9] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:28:31.118794 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-b89c59bb-e482-47d8-b8c8-dd7fa327a59a tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:28:31.118972 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-b89c59bb-e482-47d8-b8c8-dd7fa327a59a tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:28:31.119459 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-b89c59bb-e482-47d8-b8c8-dd7fa327a59a tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:28:31.120100 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-b89c59bb-e482-47d8-b8c8-dd7fa327a59a tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:28:31.120587 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-b89c59bb-e482-47d8-b8c8-dd7fa327a59a tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:28:31.121060 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-b89c59bb-e482-47d8-b8c8-dd7fa327a59a tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:28:31.171913 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [req-620f1d31-2be3-4c2a-a442-8161864ad637 req-b320786d-458d-4ea7-9822-9c16e0cf98e4 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "ac07d059-1555-4f61-9a85-28396691a934", "name": "network-vif-deleted", "tag": "42163688-0f26-433a-b60b-97256fc0be15"}]} {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 27 02:28:31.196461 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [req-620f1d31-2be3-4c2a-a442-8161864ad637 req-b320786d-458d-4ea7-9822-9c16e0cf98e4 service nova] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:28:31.198801 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [req-620f1d31-2be3-4c2a-a442-8161864ad637 req-b320786d-458d-4ea7-9822-9c16e0cf98e4 service nova] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:28:31.199528 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [req-620f1d31-2be3-4c2a-a442-8161864ad637 req-b320786d-458d-4ea7-9822-9c16e0cf98e4 service nova] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:28:31.218078 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.compute.server_external_events [req-620f1d31-2be3-4c2a-a442-8161864ad637 req-b320786d-458d-4ea7-9822-9c16e0cf98e4 service nova] Creating event network-vif-deleted:42163688-0f26-433a-b60b-97256fc0be15 for instance ac07d059-1555-4f61-9a85-28396691a934 on hv-cinder-84630 Aug 27 02:28:31.224132 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [req-620f1d31-2be3-4c2a-a442-8161864ad637 req-b320786d-458d-4ea7-9822-9c16e0cf98e4 service nova] 10.222.0.52 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.057502 Aug 27 02:28:31.225542 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 321/636] 10.222.0.52 () {64 vars in 1317 bytes} [Sat Aug 27 02:28:31 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 60 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Aug 27 02:28:31.294866 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-b89c59bb-e482-47d8-b8c8-dd7fa327a59a tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:28:31.304265 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-b89c59bb-e482-47d8-b8c8-dd7fa327a59a tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:28:31.305737 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-b89c59bb-e482-47d8-b8c8-dd7fa327a59a tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:28:31.306912 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-b89c59bb-e482-47d8-b8c8-dd7fa327a59a tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] 10.222.0.52 "GET /compute/v2.1/servers/56dbfa6a-5b6f-4b82-977b-af735e6a6ac9" status: 200 len: 1724 microversion: 2.20 time: 0.215050 Aug 27 02:28:31.307470 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 316/637] 10.222.0.52 () {60 vars in 1277 bytes} [Sat Aug 27 02:28:31 2022] GET /compute/v2.1/servers/56dbfa6a-5b6f-4b82-977b-af735e6a6ac9 => generated 1724 bytes in 216 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Aug 27 02:28:31.452739 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-4ba65086-99ef-4328-8cf6-b28e45d10ff9 tempest-VolumesSnapshotTestJSON-1750456804 tempest-VolumesSnapshotTestJSON-1750456804-project] Calling method '>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:28:31.453878 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-4ba65086-99ef-4328-8cf6-b28e45d10ff9 tempest-VolumesSnapshotTestJSON-1750456804 tempest-VolumesSnapshotTestJSON-1750456804-project] [instance: 46dae88e-3429-47c7-9376-c4c0d9cddd29] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:28:31.461392 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-4ba65086-99ef-4328-8cf6-b28e45d10ff9 tempest-VolumesSnapshotTestJSON-1750456804 tempest-VolumesSnapshotTestJSON-1750456804-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:28:31.461957 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-4ba65086-99ef-4328-8cf6-b28e45d10ff9 tempest-VolumesSnapshotTestJSON-1750456804 tempest-VolumesSnapshotTestJSON-1750456804-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:28:31.462445 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-4ba65086-99ef-4328-8cf6-b28e45d10ff9 tempest-VolumesSnapshotTestJSON-1750456804 tempest-VolumesSnapshotTestJSON-1750456804-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:28:31.463285 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-4ba65086-99ef-4328-8cf6-b28e45d10ff9 tempest-VolumesSnapshotTestJSON-1750456804 tempest-VolumesSnapshotTestJSON-1750456804-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:28:31.463784 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-4ba65086-99ef-4328-8cf6-b28e45d10ff9 tempest-VolumesSnapshotTestJSON-1750456804 tempest-VolumesSnapshotTestJSON-1750456804-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:28:31.464271 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-4ba65086-99ef-4328-8cf6-b28e45d10ff9 tempest-VolumesSnapshotTestJSON-1750456804 tempest-VolumesSnapshotTestJSON-1750456804-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:28:31.483136 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.wsgi [None req-4ba65086-99ef-4328-8cf6-b28e45d10ff9 tempest-VolumesSnapshotTestJSON-1750456804 tempest-VolumesSnapshotTestJSON-1750456804-project] HTTP exception thrown: Instance 46dae88e-3429-47c7-9376-c4c0d9cddd29 could not be found. Aug 27 02:28:31.484785 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-4ba65086-99ef-4328-8cf6-b28e45d10ff9 tempest-VolumesSnapshotTestJSON-1750456804 tempest-VolumesSnapshotTestJSON-1750456804-project] Request method failure captured: Aug 27 02:28:31.484785 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: request: GET /compute/v2.1/servers/46dae88e-3429-47c7-9376-c4c0d9cddd29 HTTP/1.1 Aug 27 02:28:31.484785 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: Accept: application/json Aug 27 02:28:31.484785 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: Accept-Encoding: identity Aug 27 02:28:31.484785 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: Connection: close Aug 27 02:28:31.484785 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: Content-Length: 0 Aug 27 02:28:31.484785 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: Content-Type: application/json Aug 27 02:28:31.484785 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: Host: 10.222.0.52 Aug 27 02:28:31.484785 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: Openstack-System-Scope: None Aug 27 02:28:31.484785 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: User-Agent: python-urllib3/1.26.11 Aug 27 02:28:31.484785 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-Auth-Token: gAAAAABjCYCjxBX_5JsN8nujRuIFBM1l5sxNDqa6fSs541Cu9nuaAmtZAqYD0aWFW_HnYB0B1tpJZv9bhJBEdRpWrr2ex3FbPNvfD0J3cVNqea_fdQysA0iZ33xjIH6pT-txE713G8wEM_gGr6-7jzRmTTMULp5yqwF_b7x9d1RaH_UV7L3KktA Aug 27 02:28:31.484785 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-Domain-Id: None Aug 27 02:28:31.484785 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-Domain-Name: None Aug 27 02:28:31.484785 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-Identity-Status: Confirmed Aug 27 02:28:31.484785 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-Is-Admin-Project: True Aug 27 02:28:31.484785 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-Project-Domain-Id: default Aug 27 02:28:31.484785 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-Project-Domain-Name: Default Aug 27 02:28:31.484785 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-Project-Id: fa43d54225bf44b283f9db5ddbbb13ca Aug 27 02:28:31.484785 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-Project-Name: tempest-VolumesSnapshotTestJSON-1750456804 Aug 27 02:28:31.484785 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-Role: reader,member Aug 27 02:28:31.484785 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-Roles: reader,member Aug 27 02:28:31.486321 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-Service-Catalog: [{"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.52:8080", "publicURL": "http://10.222.0.52:8080/v1/AUTH_fa43d54225bf44b283f9db5ddbbb13ca"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52:9696/networking"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/compute/v2/fa43d54225bf44b283f9db5ddbbb13ca"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/image"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/volume/v3/fa43d54225bf44b283f9db5ddbbb13ca"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/identity"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/volume/v3/fa43d54225bf44b283f9db5ddbbb13ca"}]}] Aug 27 02:28:31.486321 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-Tenant: tempest-VolumesSnapshotTestJSON-1750456804 Aug 27 02:28:31.486321 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-Tenant-Id: fa43d54225bf44b283f9db5ddbbb13ca Aug 27 02:28:31.486321 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-Tenant-Name: tempest-VolumesSnapshotTestJSON-1750456804 Aug 27 02:28:31.486321 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-User: tempest-VolumesSnapshotTestJSON-1750456804-project Aug 27 02:28:31.486321 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-User-Domain-Id: default Aug 27 02:28:31.486321 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-User-Domain-Name: Default Aug 27 02:28:31.486321 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-User-Id: 75744eccc3d24d4c9282cbe3a3314489 Aug 27 02:28:31.486321 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-User-Name: tempest-VolumesSnapshotTestJSON-1750456804-project Aug 27 02:28:31.486321 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: method: > Aug 27 02:28:31.486321 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: exception: Instance 46dae88e-3429-47c7-9376-c4c0d9cddd29 could not be found. Aug 27 02:28:31.486321 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: action_args: {'id': '46dae88e-3429-47c7-9376-c4c0d9cddd29'} Aug 27 02:28:31.487043 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Aug 27 02:28:31.487043 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 27 02:28:31.487043 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Aug 27 02:28:31.487043 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Aug 27 02:28:31.487043 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Aug 27 02:28:31.487043 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Aug 27 02:28:31.487043 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Aug 27 02:28:31.487043 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Aug 27 02:28:31.487043 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Aug 27 02:28:31.487043 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Aug 27 02:28:31.487043 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 46dae88e-3429-47c7-9376-c4c0d9cddd29 could not be found. Aug 27 02:28:31.487043 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi Aug 27 02:28:31.487043 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 27 02:28:31.487043 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi Aug 27 02:28:31.487043 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 27 02:28:31.487043 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 27 02:28:31.487043 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 27 02:28:31.487043 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Aug 27 02:28:31.487043 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Aug 27 02:28:31.487043 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Aug 27 02:28:31.488261 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Aug 27 02:28:31.488261 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Aug 27 02:28:31.488261 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi instance = self._get_server( Aug 27 02:28:31.488261 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Aug 27 02:28:31.488261 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Aug 27 02:28:31.488261 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Aug 27 02:28:31.488261 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Aug 27 02:28:31.488261 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 46dae88e-3429-47c7-9376-c4c0d9cddd29 could not be found. Aug 27 02:28:31.488261 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi Aug 27 02:28:31.488261 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 27 02:28:31.488261 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi Aug 27 02:28:31.488261 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 27 02:28:31.488261 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 27 02:28:31.488261 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 27 02:28:31.488261 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Aug 27 02:28:31.488261 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Aug 27 02:28:31.488261 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 46dae88e-3429-47c7-9376-c4c0d9cddd29 could not be found. Aug 27 02:28:31.488261 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi Aug 27 02:28:31.489450 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-4ba65086-99ef-4328-8cf6-b28e45d10ff9 tempest-VolumesSnapshotTestJSON-1750456804 tempest-VolumesSnapshotTestJSON-1750456804-project] Returning 404 to user: Instance 46dae88e-3429-47c7-9376-c4c0d9cddd29 could not be found. {{(pid=120864) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Aug 27 02:28:31.489450 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-4ba65086-99ef-4328-8cf6-b28e45d10ff9 tempest-VolumesSnapshotTestJSON-1750456804 tempest-VolumesSnapshotTestJSON-1750456804-project] 10.222.0.52 "GET /compute/v2.1/servers/46dae88e-3429-47c7-9376-c4c0d9cddd29" status: 404 len: 111 microversion: 2.1 time: 0.038293 Aug 27 02:28:31.489450 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 322/638] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:28:31 2022] GET /compute/v2.1/servers/46dae88e-3429-47c7-9376-c4c0d9cddd29 => generated 111 bytes in 40 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Aug 27 02:28:32.012028 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-f795a5d7-85f3-4cfd-a338-8e99bfb8dd63 tempest-VolumesNegativeTest-1839426864 tempest-VolumesNegativeTest-1839426864-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:28:32.012864 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-f795a5d7-85f3-4cfd-a338-8e99bfb8dd63 tempest-VolumesNegativeTest-1839426864 tempest-VolumesNegativeTest-1839426864-project] [instance: ac07d059-1555-4f61-9a85-28396691a934] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:28:32.019968 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-f795a5d7-85f3-4cfd-a338-8e99bfb8dd63 tempest-VolumesNegativeTest-1839426864 tempest-VolumesNegativeTest-1839426864-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:28:32.020586 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-f795a5d7-85f3-4cfd-a338-8e99bfb8dd63 tempest-VolumesNegativeTest-1839426864 tempest-VolumesNegativeTest-1839426864-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:28:32.021113 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-f795a5d7-85f3-4cfd-a338-8e99bfb8dd63 tempest-VolumesNegativeTest-1839426864 tempest-VolumesNegativeTest-1839426864-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:28:32.022327 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-f795a5d7-85f3-4cfd-a338-8e99bfb8dd63 tempest-VolumesNegativeTest-1839426864 tempest-VolumesNegativeTest-1839426864-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:28:32.022508 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-f795a5d7-85f3-4cfd-a338-8e99bfb8dd63 tempest-VolumesNegativeTest-1839426864 tempest-VolumesNegativeTest-1839426864-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:28:32.023056 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-f795a5d7-85f3-4cfd-a338-8e99bfb8dd63 tempest-VolumesNegativeTest-1839426864 tempest-VolumesNegativeTest-1839426864-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:28:32.057343 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-f795a5d7-85f3-4cfd-a338-8e99bfb8dd63 tempest-VolumesNegativeTest-1839426864 tempest-VolumesNegativeTest-1839426864-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:28:32.057848 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-f795a5d7-85f3-4cfd-a338-8e99bfb8dd63 tempest-VolumesNegativeTest-1839426864 tempest-VolumesNegativeTest-1839426864-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:28:32.058311 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-f795a5d7-85f3-4cfd-a338-8e99bfb8dd63 tempest-VolumesNegativeTest-1839426864 tempest-VolumesNegativeTest-1839426864-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:28:32.058735 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.objects.instance [None req-f795a5d7-85f3-4cfd-a338-8e99bfb8dd63 tempest-VolumesNegativeTest-1839426864 tempest-VolumesNegativeTest-1839426864-project] Lazy-loading 'fault' on Instance uuid ac07d059-1555-4f61-9a85-28396691a934 {{(pid=120863) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Aug 27 02:28:32.108541 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-f795a5d7-85f3-4cfd-a338-8e99bfb8dd63 tempest-VolumesNegativeTest-1839426864 tempest-VolumesNegativeTest-1839426864-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '544e6c056b854bfdb9e117be686635a0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d1bead2e4f4102a53f582a4815661d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:28:32.116648 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-f795a5d7-85f3-4cfd-a338-8e99bfb8dd63 tempest-VolumesNegativeTest-1839426864 tempest-VolumesNegativeTest-1839426864-project] 10.222.0.52 "GET /compute/v2.1/servers/ac07d059-1555-4f61-9a85-28396691a934" status: 200 len: 1304 microversion: 2.1 time: 0.108485 Aug 27 02:28:32.117281 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 317/639] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:28:32 2022] GET /compute/v2.1/servers/ac07d059-1555-4f61-9a85-28396691a934 => generated 1304 bytes in 110 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:28:32.335264 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-6167e676-666b-4e0a-8d25-cbf44c1ded92 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Calling method '>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:28:32.336175 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-6167e676-666b-4e0a-8d25-cbf44c1ded92 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] [instance: 56dbfa6a-5b6f-4b82-977b-af735e6a6ac9] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:28:32.348238 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-6167e676-666b-4e0a-8d25-cbf44c1ded92 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:28:32.349049 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-6167e676-666b-4e0a-8d25-cbf44c1ded92 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:28:32.349806 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-6167e676-666b-4e0a-8d25-cbf44c1ded92 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:28:32.350528 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-6167e676-666b-4e0a-8d25-cbf44c1ded92 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:28:32.350973 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-6167e676-666b-4e0a-8d25-cbf44c1ded92 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:28:32.351395 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-6167e676-666b-4e0a-8d25-cbf44c1ded92 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:28:32.496426 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-6167e676-666b-4e0a-8d25-cbf44c1ded92 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:28:32.510880 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-6167e676-666b-4e0a-8d25-cbf44c1ded92 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:28:32.513642 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-6167e676-666b-4e0a-8d25-cbf44c1ded92 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:28:32.515646 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-6167e676-666b-4e0a-8d25-cbf44c1ded92 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] 10.222.0.52 "GET /compute/v2.1/servers/56dbfa6a-5b6f-4b82-977b-af735e6a6ac9" status: 200 len: 1724 microversion: 2.20 time: 0.185064 Aug 27 02:28:32.516608 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 323/640] 10.222.0.52 () {60 vars in 1277 bytes} [Sat Aug 27 02:28:32 2022] GET /compute/v2.1/servers/56dbfa6a-5b6f-4b82-977b-af735e6a6ac9 => generated 1724 bytes in 187 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 27 02:28:33.141190 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-1b870d0a-37ea-430e-8db3-478946c3de25 tempest-VolumesNegativeTest-1839426864 tempest-VolumesNegativeTest-1839426864-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:28:33.141965 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-1b870d0a-37ea-430e-8db3-478946c3de25 tempest-VolumesNegativeTest-1839426864 tempest-VolumesNegativeTest-1839426864-project] [instance: ac07d059-1555-4f61-9a85-28396691a934] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:28:33.152634 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-1b870d0a-37ea-430e-8db3-478946c3de25 tempest-VolumesNegativeTest-1839426864 tempest-VolumesNegativeTest-1839426864-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:28:33.153503 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-1b870d0a-37ea-430e-8db3-478946c3de25 tempest-VolumesNegativeTest-1839426864 tempest-VolumesNegativeTest-1839426864-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:28:33.154127 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-1b870d0a-37ea-430e-8db3-478946c3de25 tempest-VolumesNegativeTest-1839426864 tempest-VolumesNegativeTest-1839426864-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:28:33.155360 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-1b870d0a-37ea-430e-8db3-478946c3de25 tempest-VolumesNegativeTest-1839426864 tempest-VolumesNegativeTest-1839426864-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:28:33.156018 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-1b870d0a-37ea-430e-8db3-478946c3de25 tempest-VolumesNegativeTest-1839426864 tempest-VolumesNegativeTest-1839426864-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:28:33.156765 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-1b870d0a-37ea-430e-8db3-478946c3de25 tempest-VolumesNegativeTest-1839426864 tempest-VolumesNegativeTest-1839426864-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:28:33.180208 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.wsgi [None req-1b870d0a-37ea-430e-8db3-478946c3de25 tempest-VolumesNegativeTest-1839426864 tempest-VolumesNegativeTest-1839426864-project] HTTP exception thrown: Instance ac07d059-1555-4f61-9a85-28396691a934 could not be found. Aug 27 02:28:33.183420 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-1b870d0a-37ea-430e-8db3-478946c3de25 tempest-VolumesNegativeTest-1839426864 tempest-VolumesNegativeTest-1839426864-project] Request method failure captured: Aug 27 02:28:33.183420 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: request: GET /compute/v2.1/servers/ac07d059-1555-4f61-9a85-28396691a934 HTTP/1.1 Aug 27 02:28:33.183420 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: Accept: application/json Aug 27 02:28:33.183420 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: Accept-Encoding: identity Aug 27 02:28:33.183420 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: Connection: close Aug 27 02:28:33.183420 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: Content-Length: 0 Aug 27 02:28:33.183420 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: Content-Type: application/json Aug 27 02:28:33.183420 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: Host: 10.222.0.52 Aug 27 02:28:33.183420 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: Openstack-System-Scope: None Aug 27 02:28:33.183420 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: User-Agent: python-urllib3/1.26.11 Aug 27 02:28:33.183420 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-Auth-Token: gAAAAABjCYEydQfRwQEF9jOafjgV-NZrxtdazkX5qB4Vf8gs_qHMOVgE9cIPIfiUJcznH7FRmAYBm7-8bVwcieueOa1QivsUu3BeqiEG5qSEI8ICK9IkLqhd3qOKb4Wzd7KfvOhW9r2a_4uFawmMkh07qLgGnosbmgG4VEEpz8ELRRY2_AajkfM Aug 27 02:28:33.183420 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-Domain-Id: None Aug 27 02:28:33.183420 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-Domain-Name: None Aug 27 02:28:33.183420 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-Identity-Status: Confirmed Aug 27 02:28:33.183420 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-Is-Admin-Project: True Aug 27 02:28:33.183420 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-Project-Domain-Id: default Aug 27 02:28:33.183420 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-Project-Domain-Name: Default Aug 27 02:28:33.183420 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-Project-Id: e1d1bead2e4f4102a53f582a4815661d Aug 27 02:28:33.183420 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-Project-Name: tempest-VolumesNegativeTest-1839426864 Aug 27 02:28:33.183420 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-Role: reader,member Aug 27 02:28:33.183420 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-Roles: reader,member Aug 27 02:28:33.185039 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-Service-Catalog: [{"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.52:8080", "publicURL": "http://10.222.0.52:8080/v1/AUTH_e1d1bead2e4f4102a53f582a4815661d"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52:9696/networking"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/compute/v2/e1d1bead2e4f4102a53f582a4815661d"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/image"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/volume/v3/e1d1bead2e4f4102a53f582a4815661d"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/identity"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/volume/v3/e1d1bead2e4f4102a53f582a4815661d"}]}] Aug 27 02:28:33.185039 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-Tenant: tempest-VolumesNegativeTest-1839426864 Aug 27 02:28:33.185039 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-Tenant-Id: e1d1bead2e4f4102a53f582a4815661d Aug 27 02:28:33.185039 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-Tenant-Name: tempest-VolumesNegativeTest-1839426864 Aug 27 02:28:33.185039 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-User: tempest-VolumesNegativeTest-1839426864-project Aug 27 02:28:33.185039 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-User-Domain-Id: default Aug 27 02:28:33.185039 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-User-Domain-Name: Default Aug 27 02:28:33.185039 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-User-Id: 544e6c056b854bfdb9e117be686635a0 Aug 27 02:28:33.185039 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-User-Name: tempest-VolumesNegativeTest-1839426864-project Aug 27 02:28:33.185039 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: method: > Aug 27 02:28:33.185039 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: exception: Instance ac07d059-1555-4f61-9a85-28396691a934 could not be found. Aug 27 02:28:33.185039 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: action_args: {'id': 'ac07d059-1555-4f61-9a85-28396691a934'} Aug 27 02:28:33.185983 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Aug 27 02:28:33.185983 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 27 02:28:33.185983 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Aug 27 02:28:33.185983 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Aug 27 02:28:33.185983 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Aug 27 02:28:33.185983 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Aug 27 02:28:33.185983 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Aug 27 02:28:33.185983 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Aug 27 02:28:33.185983 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Aug 27 02:28:33.185983 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Aug 27 02:28:33.185983 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance ac07d059-1555-4f61-9a85-28396691a934 could not be found. Aug 27 02:28:33.185983 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi Aug 27 02:28:33.185983 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 27 02:28:33.185983 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi Aug 27 02:28:33.185983 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 27 02:28:33.185983 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 27 02:28:33.185983 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 27 02:28:33.185983 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Aug 27 02:28:33.185983 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Aug 27 02:28:33.185983 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Aug 27 02:28:33.187348 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Aug 27 02:28:33.187348 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Aug 27 02:28:33.187348 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi instance = self._get_server( Aug 27 02:28:33.187348 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Aug 27 02:28:33.187348 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Aug 27 02:28:33.187348 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Aug 27 02:28:33.187348 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Aug 27 02:28:33.187348 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance ac07d059-1555-4f61-9a85-28396691a934 could not be found. Aug 27 02:28:33.187348 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi Aug 27 02:28:33.187348 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 27 02:28:33.187348 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi Aug 27 02:28:33.187348 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 27 02:28:33.187348 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 27 02:28:33.187348 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 27 02:28:33.187348 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Aug 27 02:28:33.187348 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Aug 27 02:28:33.187348 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance ac07d059-1555-4f61-9a85-28396691a934 could not be found. Aug 27 02:28:33.187348 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi Aug 27 02:28:33.188606 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-1b870d0a-37ea-430e-8db3-478946c3de25 tempest-VolumesNegativeTest-1839426864 tempest-VolumesNegativeTest-1839426864-project] Returning 404 to user: Instance ac07d059-1555-4f61-9a85-28396691a934 could not be found. {{(pid=120863) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Aug 27 02:28:33.188606 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-1b870d0a-37ea-430e-8db3-478946c3de25 tempest-VolumesNegativeTest-1839426864 tempest-VolumesNegativeTest-1839426864-project] 10.222.0.52 "GET /compute/v2.1/servers/ac07d059-1555-4f61-9a85-28396691a934" status: 404 len: 111 microversion: 2.1 time: 0.049249 Aug 27 02:28:33.188606 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 318/641] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:28:33 2022] GET /compute/v2.1/servers/ac07d059-1555-4f61-9a85-28396691a934 => generated 111 bytes in 52 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Aug 27 02:28:33.554781 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-ecd2dc10-542a-4b8f-8e58-bee971c9fa5c tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Calling method '>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:28:33.555460 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-ecd2dc10-542a-4b8f-8e58-bee971c9fa5c tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] [instance: 56dbfa6a-5b6f-4b82-977b-af735e6a6ac9] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:28:33.561975 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-ecd2dc10-542a-4b8f-8e58-bee971c9fa5c tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:28:33.562531 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-ecd2dc10-542a-4b8f-8e58-bee971c9fa5c tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:28:33.563032 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-ecd2dc10-542a-4b8f-8e58-bee971c9fa5c tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:28:33.563797 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-ecd2dc10-542a-4b8f-8e58-bee971c9fa5c tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:28:33.564323 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-ecd2dc10-542a-4b8f-8e58-bee971c9fa5c tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:28:33.564743 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-ecd2dc10-542a-4b8f-8e58-bee971c9fa5c tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:28:33.720281 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-ecd2dc10-542a-4b8f-8e58-bee971c9fa5c tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:28:33.731055 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-ecd2dc10-542a-4b8f-8e58-bee971c9fa5c tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:28:33.732489 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-ecd2dc10-542a-4b8f-8e58-bee971c9fa5c tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:28:33.733705 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-ecd2dc10-542a-4b8f-8e58-bee971c9fa5c tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] 10.222.0.52 "GET /compute/v2.1/servers/56dbfa6a-5b6f-4b82-977b-af735e6a6ac9" status: 200 len: 1724 microversion: 2.20 time: 0.183505 Aug 27 02:28:33.734250 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 324/642] 10.222.0.52 () {60 vars in 1277 bytes} [Sat Aug 27 02:28:33 2022] GET /compute/v2.1/servers/56dbfa6a-5b6f-4b82-977b-af735e6a6ac9 => generated 1724 bytes in 185 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 27 02:28:34.760936 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-40277952-4020-4c91-9ceb-c6495dd2de51 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:28:34.761555 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-40277952-4020-4c91-9ceb-c6495dd2de51 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] [instance: 56dbfa6a-5b6f-4b82-977b-af735e6a6ac9] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:28:34.768031 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-40277952-4020-4c91-9ceb-c6495dd2de51 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:28:34.768740 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-40277952-4020-4c91-9ceb-c6495dd2de51 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:28:34.769230 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-40277952-4020-4c91-9ceb-c6495dd2de51 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:28:34.769936 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-40277952-4020-4c91-9ceb-c6495dd2de51 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:28:34.770414 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-40277952-4020-4c91-9ceb-c6495dd2de51 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:28:34.770959 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-40277952-4020-4c91-9ceb-c6495dd2de51 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:28:34.962565 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-40277952-4020-4c91-9ceb-c6495dd2de51 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:28:34.975533 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-40277952-4020-4c91-9ceb-c6495dd2de51 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:28:34.978131 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-40277952-4020-4c91-9ceb-c6495dd2de51 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:28:34.980043 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-40277952-4020-4c91-9ceb-c6495dd2de51 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] 10.222.0.52 "GET /compute/v2.1/servers/56dbfa6a-5b6f-4b82-977b-af735e6a6ac9" status: 200 len: 1724 microversion: 2.20 time: 0.222790 Aug 27 02:28:34.980914 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 319/643] 10.222.0.52 () {60 vars in 1277 bytes} [Sat Aug 27 02:28:34 2022] GET /compute/v2.1/servers/56dbfa6a-5b6f-4b82-977b-af735e6a6ac9 => generated 1724 bytes in 225 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 27 02:28:36.014217 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-3b3289c6-6a8e-44aa-9d8f-5ecd5189d0be tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Calling method '>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:28:36.015954 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-3b3289c6-6a8e-44aa-9d8f-5ecd5189d0be tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] [instance: 56dbfa6a-5b6f-4b82-977b-af735e6a6ac9] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:28:36.028400 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-3b3289c6-6a8e-44aa-9d8f-5ecd5189d0be tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:28:36.029023 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-3b3289c6-6a8e-44aa-9d8f-5ecd5189d0be tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:28:36.029819 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-3b3289c6-6a8e-44aa-9d8f-5ecd5189d0be tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:28:36.030557 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-3b3289c6-6a8e-44aa-9d8f-5ecd5189d0be tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:28:36.031086 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-3b3289c6-6a8e-44aa-9d8f-5ecd5189d0be tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:28:36.031504 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-3b3289c6-6a8e-44aa-9d8f-5ecd5189d0be tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:28:36.231603 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-3b3289c6-6a8e-44aa-9d8f-5ecd5189d0be tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:28:36.240580 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-3b3289c6-6a8e-44aa-9d8f-5ecd5189d0be tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:28:36.242148 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-3b3289c6-6a8e-44aa-9d8f-5ecd5189d0be tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:28:36.243315 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-3b3289c6-6a8e-44aa-9d8f-5ecd5189d0be tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] 10.222.0.52 "GET /compute/v2.1/servers/56dbfa6a-5b6f-4b82-977b-af735e6a6ac9" status: 200 len: 1724 microversion: 2.20 time: 0.233248 Aug 27 02:28:36.243870 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 325/644] 10.222.0.52 () {60 vars in 1277 bytes} [Sat Aug 27 02:28:36 2022] GET /compute/v2.1/servers/56dbfa6a-5b6f-4b82-977b-af735e6a6ac9 => generated 1724 bytes in 235 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Aug 27 02:28:37.271848 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-ef1eeb5a-4e04-4861-99a7-2ea1b961049a tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:28:37.272608 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-ef1eeb5a-4e04-4861-99a7-2ea1b961049a tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] [instance: 56dbfa6a-5b6f-4b82-977b-af735e6a6ac9] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:28:37.280959 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-ef1eeb5a-4e04-4861-99a7-2ea1b961049a tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:28:37.281476 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-ef1eeb5a-4e04-4861-99a7-2ea1b961049a tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:28:37.281936 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-ef1eeb5a-4e04-4861-99a7-2ea1b961049a tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:28:37.282690 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-ef1eeb5a-4e04-4861-99a7-2ea1b961049a tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:28:37.283198 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-ef1eeb5a-4e04-4861-99a7-2ea1b961049a tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:28:37.283699 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-ef1eeb5a-4e04-4861-99a7-2ea1b961049a tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:28:37.429783 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-ef1eeb5a-4e04-4861-99a7-2ea1b961049a tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:28:37.438084 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-ef1eeb5a-4e04-4861-99a7-2ea1b961049a tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:28:37.439504 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-ef1eeb5a-4e04-4861-99a7-2ea1b961049a tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:28:37.440649 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-ef1eeb5a-4e04-4861-99a7-2ea1b961049a tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] 10.222.0.52 "GET /compute/v2.1/servers/56dbfa6a-5b6f-4b82-977b-af735e6a6ac9" status: 200 len: 1724 microversion: 2.20 time: 0.174984 Aug 27 02:28:37.441229 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 320/645] 10.222.0.52 () {60 vars in 1277 bytes} [Sat Aug 27 02:28:37 2022] GET /compute/v2.1/servers/56dbfa6a-5b6f-4b82-977b-af735e6a6ac9 => generated 1724 bytes in 176 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 27 02:28:38.468093 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-ea53f274-927c-4d9e-829e-a4af4beec165 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Calling method '>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:28:38.468884 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-ea53f274-927c-4d9e-829e-a4af4beec165 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] [instance: 56dbfa6a-5b6f-4b82-977b-af735e6a6ac9] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:28:38.476757 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-ea53f274-927c-4d9e-829e-a4af4beec165 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:28:38.477397 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-ea53f274-927c-4d9e-829e-a4af4beec165 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:28:38.477886 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-ea53f274-927c-4d9e-829e-a4af4beec165 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:28:38.478973 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-ea53f274-927c-4d9e-829e-a4af4beec165 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:28:38.479501 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-ea53f274-927c-4d9e-829e-a4af4beec165 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:28:38.479993 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-ea53f274-927c-4d9e-829e-a4af4beec165 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:28:38.666113 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-ea53f274-927c-4d9e-829e-a4af4beec165 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:28:38.682275 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-ea53f274-927c-4d9e-829e-a4af4beec165 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:28:38.685287 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-ea53f274-927c-4d9e-829e-a4af4beec165 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:28:38.687429 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-ea53f274-927c-4d9e-829e-a4af4beec165 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] 10.222.0.52 "GET /compute/v2.1/servers/56dbfa6a-5b6f-4b82-977b-af735e6a6ac9" status: 200 len: 1724 microversion: 2.20 time: 0.222938 Aug 27 02:28:38.688963 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 326/646] 10.222.0.52 () {60 vars in 1277 bytes} [Sat Aug 27 02:28:38 2022] GET /compute/v2.1/servers/56dbfa6a-5b6f-4b82-977b-af735e6a6ac9 => generated 1724 bytes in 225 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 27 02:28:39.726205 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-b3202b35-a0a6-4e19-bdb7-4999bb4a05fd tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:28:39.727253 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-b3202b35-a0a6-4e19-bdb7-4999bb4a05fd tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] [instance: 56dbfa6a-5b6f-4b82-977b-af735e6a6ac9] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:28:39.735347 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-b3202b35-a0a6-4e19-bdb7-4999bb4a05fd tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:28:39.735827 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-b3202b35-a0a6-4e19-bdb7-4999bb4a05fd tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:28:39.736257 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-b3202b35-a0a6-4e19-bdb7-4999bb4a05fd tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:28:39.737084 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-b3202b35-a0a6-4e19-bdb7-4999bb4a05fd tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:28:39.737561 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-b3202b35-a0a6-4e19-bdb7-4999bb4a05fd tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:28:39.737980 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-b3202b35-a0a6-4e19-bdb7-4999bb4a05fd tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:28:39.877383 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-b3202b35-a0a6-4e19-bdb7-4999bb4a05fd tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:28:39.886418 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-b3202b35-a0a6-4e19-bdb7-4999bb4a05fd tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:28:39.887865 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-b3202b35-a0a6-4e19-bdb7-4999bb4a05fd tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:28:39.889231 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-b3202b35-a0a6-4e19-bdb7-4999bb4a05fd tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] 10.222.0.52 "GET /compute/v2.1/servers/56dbfa6a-5b6f-4b82-977b-af735e6a6ac9" status: 200 len: 1724 microversion: 2.20 time: 0.175689 Aug 27 02:28:39.893285 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 321/647] 10.222.0.52 () {60 vars in 1277 bytes} [Sat Aug 27 02:28:39 2022] GET /compute/v2.1/servers/56dbfa6a-5b6f-4b82-977b-af735e6a6ac9 => generated 1724 bytes in 181 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 27 02:28:40.912395 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-45093c3c-74b4-4d95-9806-9c3ce738637b tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Calling method '>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:28:40.913198 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-45093c3c-74b4-4d95-9806-9c3ce738637b tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] [instance: 56dbfa6a-5b6f-4b82-977b-af735e6a6ac9] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:28:40.920518 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-45093c3c-74b4-4d95-9806-9c3ce738637b tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:28:40.921195 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-45093c3c-74b4-4d95-9806-9c3ce738637b tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:28:40.921635 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-45093c3c-74b4-4d95-9806-9c3ce738637b tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:28:40.922332 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-45093c3c-74b4-4d95-9806-9c3ce738637b tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:28:40.922787 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-45093c3c-74b4-4d95-9806-9c3ce738637b tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:28:40.923205 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-45093c3c-74b4-4d95-9806-9c3ce738637b tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:28:41.070432 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-45093c3c-74b4-4d95-9806-9c3ce738637b tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:28:41.082974 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-45093c3c-74b4-4d95-9806-9c3ce738637b tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:28:41.084691 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-45093c3c-74b4-4d95-9806-9c3ce738637b tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:28:41.086077 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-45093c3c-74b4-4d95-9806-9c3ce738637b tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] 10.222.0.52 "GET /compute/v2.1/servers/56dbfa6a-5b6f-4b82-977b-af735e6a6ac9" status: 200 len: 1745 microversion: 2.20 time: 0.177606 Aug 27 02:28:41.086670 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 327/648] 10.222.0.52 () {60 vars in 1277 bytes} [Sat Aug 27 02:28:40 2022] GET /compute/v2.1/servers/56dbfa6a-5b6f-4b82-977b-af735e6a6ac9 => generated 1745 bytes in 179 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 27 02:28:42.111557 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-957bb2d5-f79e-42e7-bec3-c9ac27d289cd tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:28:42.112233 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-957bb2d5-f79e-42e7-bec3-c9ac27d289cd tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] [instance: 56dbfa6a-5b6f-4b82-977b-af735e6a6ac9] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:28:42.121243 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-957bb2d5-f79e-42e7-bec3-c9ac27d289cd tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:28:42.122102 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-957bb2d5-f79e-42e7-bec3-c9ac27d289cd tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:28:42.122746 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-957bb2d5-f79e-42e7-bec3-c9ac27d289cd tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:28:42.124224 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-957bb2d5-f79e-42e7-bec3-c9ac27d289cd tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:28:42.124681 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-957bb2d5-f79e-42e7-bec3-c9ac27d289cd tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:28:42.125145 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-957bb2d5-f79e-42e7-bec3-c9ac27d289cd tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:28:42.266174 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-957bb2d5-f79e-42e7-bec3-c9ac27d289cd tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:28:42.278085 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-957bb2d5-f79e-42e7-bec3-c9ac27d289cd tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:28:42.279700 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-957bb2d5-f79e-42e7-bec3-c9ac27d289cd tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:28:42.281035 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-957bb2d5-f79e-42e7-bec3-c9ac27d289cd tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] 10.222.0.52 "GET /compute/v2.1/servers/56dbfa6a-5b6f-4b82-977b-af735e6a6ac9" status: 200 len: 1745 microversion: 2.20 time: 0.174369 Aug 27 02:28:42.281653 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 322/649] 10.222.0.52 () {60 vars in 1277 bytes} [Sat Aug 27 02:28:42 2022] GET /compute/v2.1/servers/56dbfa6a-5b6f-4b82-977b-af735e6a6ac9 => generated 1745 bytes in 176 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 27 02:28:43.321863 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-783012e3-37f5-4c64-bea8-55cdda775e7c tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Calling method '>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:28:43.322582 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-783012e3-37f5-4c64-bea8-55cdda775e7c tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] [instance: 56dbfa6a-5b6f-4b82-977b-af735e6a6ac9] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:28:43.333340 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-783012e3-37f5-4c64-bea8-55cdda775e7c tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:28:43.334603 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-783012e3-37f5-4c64-bea8-55cdda775e7c tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:28:43.335393 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-783012e3-37f5-4c64-bea8-55cdda775e7c tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:28:43.336507 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-783012e3-37f5-4c64-bea8-55cdda775e7c tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:28:43.337106 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-783012e3-37f5-4c64-bea8-55cdda775e7c tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:28:43.337618 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-783012e3-37f5-4c64-bea8-55cdda775e7c tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:28:43.492049 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-783012e3-37f5-4c64-bea8-55cdda775e7c tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:28:43.504306 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-783012e3-37f5-4c64-bea8-55cdda775e7c tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:28:43.506567 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-783012e3-37f5-4c64-bea8-55cdda775e7c tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:28:43.508140 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-783012e3-37f5-4c64-bea8-55cdda775e7c tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] 10.222.0.52 "GET /compute/v2.1/servers/56dbfa6a-5b6f-4b82-977b-af735e6a6ac9" status: 200 len: 1745 microversion: 2.20 time: 0.192532 Aug 27 02:28:43.525171 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 328/650] 10.222.0.52 () {60 vars in 1277 bytes} [Sat Aug 27 02:28:43 2022] GET /compute/v2.1/servers/56dbfa6a-5b6f-4b82-977b-af735e6a6ac9 => generated 1745 bytes in 201 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Aug 27 02:28:43.696094 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [req-bafea047-bde9-4923-b322-c4653a4a3d96 req-106e16e8-7b92-4c13-995e-30e473efc35a service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "56dbfa6a-5b6f-4b82-977b-af735e6a6ac9", "name": "network-vif-plugged", "status": "completed", "tag": "520bd5fb-723a-4570-83ca-72759c588707"}]} {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 27 02:28:43.701776 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG dbcounter [-] [120863] Writing DB stats nova_api:SELECT=68,nova_api:UPDATE=2 {{(pid=120863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 27 02:28:43.708042 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [req-bafea047-bde9-4923-b322-c4653a4a3d96 req-106e16e8-7b92-4c13-995e-30e473efc35a service nova] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:28:43.708569 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [req-bafea047-bde9-4923-b322-c4653a4a3d96 req-106e16e8-7b92-4c13-995e-30e473efc35a service nova] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:28:43.709268 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [req-bafea047-bde9-4923-b322-c4653a4a3d96 req-106e16e8-7b92-4c13-995e-30e473efc35a service nova] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:28:43.722673 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.compute.server_external_events [req-bafea047-bde9-4923-b322-c4653a4a3d96 req-106e16e8-7b92-4c13-995e-30e473efc35a service nova] Creating event network-vif-plugged:520bd5fb-723a-4570-83ca-72759c588707 for instance 56dbfa6a-5b6f-4b82-977b-af735e6a6ac9 on hv-cinder-84630 Aug 27 02:28:43.728331 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [req-bafea047-bde9-4923-b322-c4653a4a3d96 req-106e16e8-7b92-4c13-995e-30e473efc35a service nova] 10.222.0.52 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.275189 Aug 27 02:28:43.728983 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 323/651] 10.222.0.52 () {64 vars in 1317 bytes} [Sat Aug 27 02:28:43 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 277 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 27 02:28:44.538477 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-98aefa05-7584-42a2-b715-95b87b05b3cb tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Calling method '>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:28:44.539050 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-98aefa05-7584-42a2-b715-95b87b05b3cb tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] [instance: 56dbfa6a-5b6f-4b82-977b-af735e6a6ac9] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:28:44.541429 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG dbcounter [-] [120864] Writing DB stats nova_api:SELECT=64 {{(pid=120864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 27 02:28:44.547095 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-98aefa05-7584-42a2-b715-95b87b05b3cb tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:28:44.547589 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-98aefa05-7584-42a2-b715-95b87b05b3cb tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:28:44.548051 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-98aefa05-7584-42a2-b715-95b87b05b3cb tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:28:44.548725 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-98aefa05-7584-42a2-b715-95b87b05b3cb tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:28:44.549198 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-98aefa05-7584-42a2-b715-95b87b05b3cb tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:28:44.549622 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-98aefa05-7584-42a2-b715-95b87b05b3cb tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:28:44.707679 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-98aefa05-7584-42a2-b715-95b87b05b3cb tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:28:44.717103 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-98aefa05-7584-42a2-b715-95b87b05b3cb tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:28:44.717945 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-98aefa05-7584-42a2-b715-95b87b05b3cb tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:28:44.719297 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-98aefa05-7584-42a2-b715-95b87b05b3cb tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] 10.222.0.52 "GET /compute/v2.1/servers/56dbfa6a-5b6f-4b82-977b-af735e6a6ac9" status: 200 len: 1745 microversion: 2.20 time: 0.186351 Aug 27 02:28:44.719848 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 329/652] 10.222.0.52 () {60 vars in 1277 bytes} [Sat Aug 27 02:28:44 2022] GET /compute/v2.1/servers/56dbfa6a-5b6f-4b82-977b-af735e6a6ac9 => generated 1745 bytes in 188 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 27 02:28:45.743742 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-2e429ace-9559-4684-b72a-b3f1b48f348f tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:28:45.744586 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-2e429ace-9559-4684-b72a-b3f1b48f348f tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] [instance: 56dbfa6a-5b6f-4b82-977b-af735e6a6ac9] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:28:45.751081 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-2e429ace-9559-4684-b72a-b3f1b48f348f tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:28:45.751576 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-2e429ace-9559-4684-b72a-b3f1b48f348f tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:28:45.752029 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-2e429ace-9559-4684-b72a-b3f1b48f348f tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:28:45.752740 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-2e429ace-9559-4684-b72a-b3f1b48f348f tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:28:45.753206 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-2e429ace-9559-4684-b72a-b3f1b48f348f tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:28:45.753634 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-2e429ace-9559-4684-b72a-b3f1b48f348f tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:28:45.893077 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-2e429ace-9559-4684-b72a-b3f1b48f348f tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:28:45.906134 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-2e429ace-9559-4684-b72a-b3f1b48f348f tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:28:45.907753 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-2e429ace-9559-4684-b72a-b3f1b48f348f tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:28:45.908917 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-2e429ace-9559-4684-b72a-b3f1b48f348f tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] 10.222.0.52 "GET /compute/v2.1/servers/56dbfa6a-5b6f-4b82-977b-af735e6a6ac9" status: 200 len: 1745 microversion: 2.20 time: 0.170305 Aug 27 02:28:45.909632 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 324/653] 10.222.0.52 () {60 vars in 1277 bytes} [Sat Aug 27 02:28:45 2022] GET /compute/v2.1/servers/56dbfa6a-5b6f-4b82-977b-af735e6a6ac9 => generated 1745 bytes in 172 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Aug 27 02:28:46.933668 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-09c19eec-1523-4ce7-b6a6-ca1deff590f7 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Calling method '>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:28:46.934242 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-09c19eec-1523-4ce7-b6a6-ca1deff590f7 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] [instance: 56dbfa6a-5b6f-4b82-977b-af735e6a6ac9] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:28:46.941506 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-09c19eec-1523-4ce7-b6a6-ca1deff590f7 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:28:46.941985 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-09c19eec-1523-4ce7-b6a6-ca1deff590f7 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:28:46.942416 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-09c19eec-1523-4ce7-b6a6-ca1deff590f7 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:28:46.943116 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-09c19eec-1523-4ce7-b6a6-ca1deff590f7 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:28:46.943565 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-09c19eec-1523-4ce7-b6a6-ca1deff590f7 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:28:46.943981 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-09c19eec-1523-4ce7-b6a6-ca1deff590f7 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:28:47.075096 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-09c19eec-1523-4ce7-b6a6-ca1deff590f7 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:28:47.083693 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-09c19eec-1523-4ce7-b6a6-ca1deff590f7 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:28:47.085102 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-09c19eec-1523-4ce7-b6a6-ca1deff590f7 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:28:47.086281 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-09c19eec-1523-4ce7-b6a6-ca1deff590f7 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] 10.222.0.52 "GET /compute/v2.1/servers/56dbfa6a-5b6f-4b82-977b-af735e6a6ac9" status: 200 len: 1745 microversion: 2.20 time: 0.156584 Aug 27 02:28:47.086831 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 330/654] 10.222.0.52 () {60 vars in 1277 bytes} [Sat Aug 27 02:28:46 2022] GET /compute/v2.1/servers/56dbfa6a-5b6f-4b82-977b-af735e6a6ac9 => generated 1745 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 27 02:28:48.110041 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-c7e092d0-ee7a-4670-8718-a05c5b1b72a0 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:28:48.110632 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-c7e092d0-ee7a-4670-8718-a05c5b1b72a0 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] [instance: 56dbfa6a-5b6f-4b82-977b-af735e6a6ac9] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:28:48.118344 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-c7e092d0-ee7a-4670-8718-a05c5b1b72a0 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:28:48.118832 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-c7e092d0-ee7a-4670-8718-a05c5b1b72a0 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:28:48.119272 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-c7e092d0-ee7a-4670-8718-a05c5b1b72a0 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:28:48.119978 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-c7e092d0-ee7a-4670-8718-a05c5b1b72a0 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:28:48.120450 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-c7e092d0-ee7a-4670-8718-a05c5b1b72a0 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:28:48.120873 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-c7e092d0-ee7a-4670-8718-a05c5b1b72a0 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:28:48.264927 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-c7e092d0-ee7a-4670-8718-a05c5b1b72a0 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:28:48.275864 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-c7e092d0-ee7a-4670-8718-a05c5b1b72a0 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:28:48.277362 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-c7e092d0-ee7a-4670-8718-a05c5b1b72a0 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:28:48.278535 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-c7e092d0-ee7a-4670-8718-a05c5b1b72a0 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] 10.222.0.52 "GET /compute/v2.1/servers/56dbfa6a-5b6f-4b82-977b-af735e6a6ac9" status: 200 len: 1745 microversion: 2.20 time: 0.172486 Aug 27 02:28:48.279120 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 325/655] 10.222.0.52 () {60 vars in 1277 bytes} [Sat Aug 27 02:28:48 2022] GET /compute/v2.1/servers/56dbfa6a-5b6f-4b82-977b-af735e6a6ac9 => generated 1745 bytes in 174 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 27 02:28:49.302244 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-8f45a62a-8199-4e43-9843-e256133828d3 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Calling method '>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:28:49.302854 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-8f45a62a-8199-4e43-9843-e256133828d3 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] [instance: 56dbfa6a-5b6f-4b82-977b-af735e6a6ac9] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:28:49.310250 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-8f45a62a-8199-4e43-9843-e256133828d3 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:28:49.310731 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-8f45a62a-8199-4e43-9843-e256133828d3 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:28:49.311262 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-8f45a62a-8199-4e43-9843-e256133828d3 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:28:49.311958 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-8f45a62a-8199-4e43-9843-e256133828d3 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:28:49.312406 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-8f45a62a-8199-4e43-9843-e256133828d3 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:28:49.312826 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-8f45a62a-8199-4e43-9843-e256133828d3 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:28:49.315695 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG dbcounter [-] [120864] Writing DB stats nova_cell1:SELECT=100 {{(pid=120864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 27 02:28:49.469792 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-8f45a62a-8199-4e43-9843-e256133828d3 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:28:49.478492 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-8f45a62a-8199-4e43-9843-e256133828d3 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:28:49.479961 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-8f45a62a-8199-4e43-9843-e256133828d3 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:28:49.481245 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-8f45a62a-8199-4e43-9843-e256133828d3 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] 10.222.0.52 "GET /compute/v2.1/servers/56dbfa6a-5b6f-4b82-977b-af735e6a6ac9" status: 200 len: 1745 microversion: 2.20 time: 0.183301 Aug 27 02:28:49.481799 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 331/656] 10.222.0.52 () {60 vars in 1277 bytes} [Sat Aug 27 02:28:49 2022] GET /compute/v2.1/servers/56dbfa6a-5b6f-4b82-977b-af735e6a6ac9 => generated 1745 bytes in 185 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 27 02:28:50.510321 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-259e29ae-4593-4c75-a492-cc52acea89bb tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:28:50.510972 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-259e29ae-4593-4c75-a492-cc52acea89bb tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] [instance: 56dbfa6a-5b6f-4b82-977b-af735e6a6ac9] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:28:50.517783 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-259e29ae-4593-4c75-a492-cc52acea89bb tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:28:50.518276 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-259e29ae-4593-4c75-a492-cc52acea89bb tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:28:50.518724 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-259e29ae-4593-4c75-a492-cc52acea89bb tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:28:50.519413 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-259e29ae-4593-4c75-a492-cc52acea89bb tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:28:50.519904 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-259e29ae-4593-4c75-a492-cc52acea89bb tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:28:50.520526 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-259e29ae-4593-4c75-a492-cc52acea89bb tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:28:50.679295 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-259e29ae-4593-4c75-a492-cc52acea89bb tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:28:50.681588 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG dbcounter [-] [120863] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=120863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 27 02:28:50.688749 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-259e29ae-4593-4c75-a492-cc52acea89bb tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:28:50.690208 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-259e29ae-4593-4c75-a492-cc52acea89bb tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:28:50.691376 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-259e29ae-4593-4c75-a492-cc52acea89bb tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] 10.222.0.52 "GET /compute/v2.1/servers/56dbfa6a-5b6f-4b82-977b-af735e6a6ac9" status: 200 len: 1745 microversion: 2.20 time: 0.185105 Aug 27 02:28:50.691917 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 326/657] 10.222.0.52 () {60 vars in 1277 bytes} [Sat Aug 27 02:28:50 2022] GET /compute/v2.1/servers/56dbfa6a-5b6f-4b82-977b-af735e6a6ac9 => generated 1745 bytes in 186 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Aug 27 02:28:51.716336 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-1904bb32-73fe-42fe-884a-e37bf1cb47c9 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Calling method '>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:28:51.717193 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-1904bb32-73fe-42fe-884a-e37bf1cb47c9 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] [instance: 56dbfa6a-5b6f-4b82-977b-af735e6a6ac9] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:28:51.724407 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-1904bb32-73fe-42fe-884a-e37bf1cb47c9 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:28:51.724889 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-1904bb32-73fe-42fe-884a-e37bf1cb47c9 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:28:51.725528 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-1904bb32-73fe-42fe-884a-e37bf1cb47c9 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:28:51.726227 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-1904bb32-73fe-42fe-884a-e37bf1cb47c9 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:28:51.726666 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-1904bb32-73fe-42fe-884a-e37bf1cb47c9 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:28:51.727082 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-1904bb32-73fe-42fe-884a-e37bf1cb47c9 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:28:51.858837 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-1904bb32-73fe-42fe-884a-e37bf1cb47c9 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:28:51.868341 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-1904bb32-73fe-42fe-884a-e37bf1cb47c9 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:28:51.869858 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-1904bb32-73fe-42fe-884a-e37bf1cb47c9 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:28:51.871062 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-1904bb32-73fe-42fe-884a-e37bf1cb47c9 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] 10.222.0.52 "GET /compute/v2.1/servers/56dbfa6a-5b6f-4b82-977b-af735e6a6ac9" status: 200 len: 1745 microversion: 2.20 time: 0.158684 Aug 27 02:28:51.871901 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 332/658] 10.222.0.52 () {60 vars in 1277 bytes} [Sat Aug 27 02:28:51 2022] GET /compute/v2.1/servers/56dbfa6a-5b6f-4b82-977b-af735e6a6ac9 => generated 1745 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 27 02:28:52.906692 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-e07db323-75a7-4c75-b3e6-dc510887c92f tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:28:52.907420 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-e07db323-75a7-4c75-b3e6-dc510887c92f tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] [instance: 56dbfa6a-5b6f-4b82-977b-af735e6a6ac9] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:28:52.916732 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-e07db323-75a7-4c75-b3e6-dc510887c92f tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:28:52.917253 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-e07db323-75a7-4c75-b3e6-dc510887c92f tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:28:52.917762 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-e07db323-75a7-4c75-b3e6-dc510887c92f tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:28:52.918488 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-e07db323-75a7-4c75-b3e6-dc510887c92f tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:28:52.918961 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-e07db323-75a7-4c75-b3e6-dc510887c92f tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:28:52.919411 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-e07db323-75a7-4c75-b3e6-dc510887c92f tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:28:53.088755 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-e07db323-75a7-4c75-b3e6-dc510887c92f tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:28:53.097058 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-e07db323-75a7-4c75-b3e6-dc510887c92f tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:28:53.098485 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-e07db323-75a7-4c75-b3e6-dc510887c92f tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:28:53.099641 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-e07db323-75a7-4c75-b3e6-dc510887c92f tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] 10.222.0.52 "GET /compute/v2.1/servers/56dbfa6a-5b6f-4b82-977b-af735e6a6ac9" status: 200 len: 1740 microversion: 2.20 time: 0.197402 Aug 27 02:28:53.100870 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 327/659] 10.222.0.52 () {60 vars in 1277 bytes} [Sat Aug 27 02:28:52 2022] GET /compute/v2.1/servers/56dbfa6a-5b6f-4b82-977b-af735e6a6ac9 => generated 1740 bytes in 199 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Aug 27 02:28:54.130222 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-0e44f87d-8201-4252-bc90-51986cef137a tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Calling method '>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:28:54.130788 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-0e44f87d-8201-4252-bc90-51986cef137a tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] [instance: 56dbfa6a-5b6f-4b82-977b-af735e6a6ac9] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:28:54.137893 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-0e44f87d-8201-4252-bc90-51986cef137a tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:28:54.138364 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-0e44f87d-8201-4252-bc90-51986cef137a tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:28:54.138799 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-0e44f87d-8201-4252-bc90-51986cef137a tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:28:54.139501 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-0e44f87d-8201-4252-bc90-51986cef137a tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:28:54.140005 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-0e44f87d-8201-4252-bc90-51986cef137a tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:28:54.140448 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-0e44f87d-8201-4252-bc90-51986cef137a tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:28:54.319963 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-0e44f87d-8201-4252-bc90-51986cef137a tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:28:54.332095 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-0e44f87d-8201-4252-bc90-51986cef137a tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:28:54.333836 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-0e44f87d-8201-4252-bc90-51986cef137a tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:28:54.335346 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-0e44f87d-8201-4252-bc90-51986cef137a tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] 10.222.0.52 "GET /compute/v2.1/servers/56dbfa6a-5b6f-4b82-977b-af735e6a6ac9" status: 200 len: 1740 microversion: 2.20 time: 0.208821 Aug 27 02:28:54.335985 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 333/660] 10.222.0.52 () {60 vars in 1277 bytes} [Sat Aug 27 02:28:54 2022] GET /compute/v2.1/servers/56dbfa6a-5b6f-4b82-977b-af735e6a6ac9 => generated 1740 bytes in 210 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 27 02:28:55.359534 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-84f7e5d1-1a08-46a9-b0cf-407464a4272e tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:28:55.360499 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-84f7e5d1-1a08-46a9-b0cf-407464a4272e tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] [instance: 56dbfa6a-5b6f-4b82-977b-af735e6a6ac9] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:28:55.367566 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-84f7e5d1-1a08-46a9-b0cf-407464a4272e tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:28:55.368174 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-84f7e5d1-1a08-46a9-b0cf-407464a4272e tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:28:55.368812 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-84f7e5d1-1a08-46a9-b0cf-407464a4272e tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:28:55.369743 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-84f7e5d1-1a08-46a9-b0cf-407464a4272e tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:28:55.370199 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-84f7e5d1-1a08-46a9-b0cf-407464a4272e tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:28:55.370766 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-84f7e5d1-1a08-46a9-b0cf-407464a4272e tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:28:55.562973 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-84f7e5d1-1a08-46a9-b0cf-407464a4272e tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:28:55.580612 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-84f7e5d1-1a08-46a9-b0cf-407464a4272e tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:28:55.582110 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-84f7e5d1-1a08-46a9-b0cf-407464a4272e tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:28:55.583603 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-84f7e5d1-1a08-46a9-b0cf-407464a4272e tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] 10.222.0.52 "GET /compute/v2.1/servers/56dbfa6a-5b6f-4b82-977b-af735e6a6ac9" status: 200 len: 1740 microversion: 2.20 time: 0.228116 Aug 27 02:28:55.584669 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 328/661] 10.222.0.52 () {60 vars in 1277 bytes} [Sat Aug 27 02:28:55 2022] GET /compute/v2.1/servers/56dbfa6a-5b6f-4b82-977b-af735e6a6ac9 => generated 1740 bytes in 230 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 27 02:28:56.618474 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-bcc2c81b-7d4a-44a5-a281-fb91c44ac8f1 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Calling method '>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:28:56.619144 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-bcc2c81b-7d4a-44a5-a281-fb91c44ac8f1 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] [instance: 56dbfa6a-5b6f-4b82-977b-af735e6a6ac9] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:28:56.638032 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-bcc2c81b-7d4a-44a5-a281-fb91c44ac8f1 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:28:56.638933 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-bcc2c81b-7d4a-44a5-a281-fb91c44ac8f1 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:28:56.639610 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-bcc2c81b-7d4a-44a5-a281-fb91c44ac8f1 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:28:56.640379 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-bcc2c81b-7d4a-44a5-a281-fb91c44ac8f1 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:28:56.640830 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-bcc2c81b-7d4a-44a5-a281-fb91c44ac8f1 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:28:56.641284 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-bcc2c81b-7d4a-44a5-a281-fb91c44ac8f1 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:28:56.800741 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-bcc2c81b-7d4a-44a5-a281-fb91c44ac8f1 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:28:56.815755 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-bcc2c81b-7d4a-44a5-a281-fb91c44ac8f1 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:28:56.818239 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-bcc2c81b-7d4a-44a5-a281-fb91c44ac8f1 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:28:56.820837 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-bcc2c81b-7d4a-44a5-a281-fb91c44ac8f1 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] 10.222.0.52 "GET /compute/v2.1/servers/56dbfa6a-5b6f-4b82-977b-af735e6a6ac9" status: 200 len: 1740 microversion: 2.20 time: 0.207762 Aug 27 02:28:56.821950 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 334/662] 10.222.0.52 () {60 vars in 1277 bytes} [Sat Aug 27 02:28:56 2022] GET /compute/v2.1/servers/56dbfa6a-5b6f-4b82-977b-af735e6a6ac9 => generated 1740 bytes in 210 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 27 02:28:57.857719 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-a0c63f87-23e8-4f0f-b1f2-70871df4909b tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:28:57.858680 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-a0c63f87-23e8-4f0f-b1f2-70871df4909b tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] [instance: 56dbfa6a-5b6f-4b82-977b-af735e6a6ac9] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:28:57.870974 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-a0c63f87-23e8-4f0f-b1f2-70871df4909b tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:28:57.871654 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-a0c63f87-23e8-4f0f-b1f2-70871df4909b tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:28:57.872291 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-a0c63f87-23e8-4f0f-b1f2-70871df4909b tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:28:57.873333 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-a0c63f87-23e8-4f0f-b1f2-70871df4909b tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:28:57.874305 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-a0c63f87-23e8-4f0f-b1f2-70871df4909b tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:28:57.874992 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-a0c63f87-23e8-4f0f-b1f2-70871df4909b tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:28:58.087927 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-a0c63f87-23e8-4f0f-b1f2-70871df4909b tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:28:58.098747 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-a0c63f87-23e8-4f0f-b1f2-70871df4909b tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:28:58.100825 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-a0c63f87-23e8-4f0f-b1f2-70871df4909b tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:28:58.102292 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-a0c63f87-23e8-4f0f-b1f2-70871df4909b tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] 10.222.0.52 "GET /compute/v2.1/servers/56dbfa6a-5b6f-4b82-977b-af735e6a6ac9" status: 200 len: 1740 microversion: 2.20 time: 0.250711 Aug 27 02:28:58.103070 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 329/663] 10.222.0.52 () {60 vars in 1277 bytes} [Sat Aug 27 02:28:57 2022] GET /compute/v2.1/servers/56dbfa6a-5b6f-4b82-977b-af735e6a6ac9 => generated 1740 bytes in 253 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 27 02:28:59.128435 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-8a891fbf-3355-4476-b9b8-b222a36e6fd0 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Calling method '>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:28:59.134653 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-8a891fbf-3355-4476-b9b8-b222a36e6fd0 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] [instance: 56dbfa6a-5b6f-4b82-977b-af735e6a6ac9] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:28:59.144388 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-8a891fbf-3355-4476-b9b8-b222a36e6fd0 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:28:59.145360 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-8a891fbf-3355-4476-b9b8-b222a36e6fd0 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:28:59.146031 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-8a891fbf-3355-4476-b9b8-b222a36e6fd0 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:28:59.147034 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-8a891fbf-3355-4476-b9b8-b222a36e6fd0 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:28:59.148285 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-8a891fbf-3355-4476-b9b8-b222a36e6fd0 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:28:59.148833 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-8a891fbf-3355-4476-b9b8-b222a36e6fd0 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:28:59.361501 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-8a891fbf-3355-4476-b9b8-b222a36e6fd0 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:28:59.375557 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-8a891fbf-3355-4476-b9b8-b222a36e6fd0 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:28:59.378427 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-8a891fbf-3355-4476-b9b8-b222a36e6fd0 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:28:59.380560 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-8a891fbf-3355-4476-b9b8-b222a36e6fd0 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] 10.222.0.52 "GET /compute/v2.1/servers/56dbfa6a-5b6f-4b82-977b-af735e6a6ac9" status: 200 len: 1740 microversion: 2.20 time: 0.257622 Aug 27 02:28:59.381770 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 335/664] 10.222.0.52 () {60 vars in 1277 bytes} [Sat Aug 27 02:28:59 2022] GET /compute/v2.1/servers/56dbfa6a-5b6f-4b82-977b-af735e6a6ac9 => generated 1740 bytes in 260 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Aug 27 02:29:00.412919 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-fed9998b-c63f-4c80-a081-857bd4b778f8 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:29:00.416306 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-fed9998b-c63f-4c80-a081-857bd4b778f8 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] [instance: 56dbfa6a-5b6f-4b82-977b-af735e6a6ac9] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:29:00.430213 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-fed9998b-c63f-4c80-a081-857bd4b778f8 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:29:00.431586 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-fed9998b-c63f-4c80-a081-857bd4b778f8 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:29:00.432573 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-fed9998b-c63f-4c80-a081-857bd4b778f8 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:29:00.435377 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-fed9998b-c63f-4c80-a081-857bd4b778f8 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:29:00.436415 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-fed9998b-c63f-4c80-a081-857bd4b778f8 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:29:00.436840 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-fed9998b-c63f-4c80-a081-857bd4b778f8 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:29:00.631878 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-fed9998b-c63f-4c80-a081-857bd4b778f8 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:29:00.640477 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-fed9998b-c63f-4c80-a081-857bd4b778f8 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:29:00.642443 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-fed9998b-c63f-4c80-a081-857bd4b778f8 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:29:00.643846 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-fed9998b-c63f-4c80-a081-857bd4b778f8 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] 10.222.0.52 "GET /compute/v2.1/servers/56dbfa6a-5b6f-4b82-977b-af735e6a6ac9" status: 200 len: 1740 microversion: 2.20 time: 0.240621 Aug 27 02:29:00.644593 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 330/665] 10.222.0.52 () {60 vars in 1277 bytes} [Sat Aug 27 02:29:00 2022] GET /compute/v2.1/servers/56dbfa6a-5b6f-4b82-977b-af735e6a6ac9 => generated 1740 bytes in 243 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 27 02:29:01.668450 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-b2d23b06-c3f4-42a0-bc13-8227839259f8 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Calling method '>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:29:01.669173 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-b2d23b06-c3f4-42a0-bc13-8227839259f8 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] [instance: 56dbfa6a-5b6f-4b82-977b-af735e6a6ac9] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:29:01.684315 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-b2d23b06-c3f4-42a0-bc13-8227839259f8 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:29:01.684812 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-b2d23b06-c3f4-42a0-bc13-8227839259f8 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:29:01.685277 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-b2d23b06-c3f4-42a0-bc13-8227839259f8 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:29:01.685976 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-b2d23b06-c3f4-42a0-bc13-8227839259f8 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:29:01.686606 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-b2d23b06-c3f4-42a0-bc13-8227839259f8 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:29:01.687393 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-b2d23b06-c3f4-42a0-bc13-8227839259f8 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:29:01.895156 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-b2d23b06-c3f4-42a0-bc13-8227839259f8 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:29:01.904505 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-b2d23b06-c3f4-42a0-bc13-8227839259f8 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:29:01.906394 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-b2d23b06-c3f4-42a0-bc13-8227839259f8 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:29:01.908021 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-b2d23b06-c3f4-42a0-bc13-8227839259f8 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] 10.222.0.52 "GET /compute/v2.1/servers/56dbfa6a-5b6f-4b82-977b-af735e6a6ac9" status: 200 len: 1740 microversion: 2.20 time: 0.243725 Aug 27 02:29:01.909278 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 336/666] 10.222.0.52 () {60 vars in 1277 bytes} [Sat Aug 27 02:29:01 2022] GET /compute/v2.1/servers/56dbfa6a-5b6f-4b82-977b-af735e6a6ac9 => generated 1740 bytes in 245 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 27 02:29:02.952771 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-7d67a83b-d912-4f0a-aac2-431f77e784ad tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:29:02.953993 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-7d67a83b-d912-4f0a-aac2-431f77e784ad tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] [instance: 56dbfa6a-5b6f-4b82-977b-af735e6a6ac9] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:29:02.964148 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-7d67a83b-d912-4f0a-aac2-431f77e784ad tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:29:02.964640 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-7d67a83b-d912-4f0a-aac2-431f77e784ad tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:29:02.965279 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-7d67a83b-d912-4f0a-aac2-431f77e784ad tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:29:02.966240 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-7d67a83b-d912-4f0a-aac2-431f77e784ad tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:29:02.967310 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-7d67a83b-d912-4f0a-aac2-431f77e784ad tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:29:02.967786 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-7d67a83b-d912-4f0a-aac2-431f77e784ad tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:29:03.191235 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-7d67a83b-d912-4f0a-aac2-431f77e784ad tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:29:03.200804 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-7d67a83b-d912-4f0a-aac2-431f77e784ad tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:29:03.202273 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-7d67a83b-d912-4f0a-aac2-431f77e784ad tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:29:03.203662 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-7d67a83b-d912-4f0a-aac2-431f77e784ad tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] 10.222.0.52 "GET /compute/v2.1/servers/56dbfa6a-5b6f-4b82-977b-af735e6a6ac9" status: 200 len: 1740 microversion: 2.20 time: 0.258195 Aug 27 02:29:03.204217 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 331/667] 10.222.0.52 () {60 vars in 1277 bytes} [Sat Aug 27 02:29:02 2022] GET /compute/v2.1/servers/56dbfa6a-5b6f-4b82-977b-af735e6a6ac9 => generated 1740 bytes in 261 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 27 02:29:04.238877 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-f73fc4da-e11a-4dd2-a858-786b7e8ccd82 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Calling method '>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:29:04.239473 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-f73fc4da-e11a-4dd2-a858-786b7e8ccd82 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] [instance: 56dbfa6a-5b6f-4b82-977b-af735e6a6ac9] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:29:04.246641 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-f73fc4da-e11a-4dd2-a858-786b7e8ccd82 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:29:04.247126 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-f73fc4da-e11a-4dd2-a858-786b7e8ccd82 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:29:04.247568 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-f73fc4da-e11a-4dd2-a858-786b7e8ccd82 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:29:04.248258 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-f73fc4da-e11a-4dd2-a858-786b7e8ccd82 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:29:04.248698 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-f73fc4da-e11a-4dd2-a858-786b7e8ccd82 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:29:04.249158 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-f73fc4da-e11a-4dd2-a858-786b7e8ccd82 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:29:04.421500 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-f73fc4da-e11a-4dd2-a858-786b7e8ccd82 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:29:04.438553 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-f73fc4da-e11a-4dd2-a858-786b7e8ccd82 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:29:04.440453 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-f73fc4da-e11a-4dd2-a858-786b7e8ccd82 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:29:04.441921 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-f73fc4da-e11a-4dd2-a858-786b7e8ccd82 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] 10.222.0.52 "GET /compute/v2.1/servers/56dbfa6a-5b6f-4b82-977b-af735e6a6ac9" status: 200 len: 1740 microversion: 2.20 time: 0.207060 Aug 27 02:29:04.442743 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 337/668] 10.222.0.52 () {60 vars in 1277 bytes} [Sat Aug 27 02:29:04 2022] GET /compute/v2.1/servers/56dbfa6a-5b6f-4b82-977b-af735e6a6ac9 => generated 1740 bytes in 209 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Aug 27 02:29:05.474576 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-396e5a5b-18c7-4f04-b043-36c1210037b4 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:29:05.476018 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-396e5a5b-18c7-4f04-b043-36c1210037b4 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] [instance: 56dbfa6a-5b6f-4b82-977b-af735e6a6ac9] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:29:05.485598 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-396e5a5b-18c7-4f04-b043-36c1210037b4 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:29:05.486284 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-396e5a5b-18c7-4f04-b043-36c1210037b4 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:29:05.486855 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-396e5a5b-18c7-4f04-b043-36c1210037b4 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:29:05.487676 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-396e5a5b-18c7-4f04-b043-36c1210037b4 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:29:05.488248 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-396e5a5b-18c7-4f04-b043-36c1210037b4 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:29:05.488844 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-396e5a5b-18c7-4f04-b043-36c1210037b4 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:29:05.675885 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-396e5a5b-18c7-4f04-b043-36c1210037b4 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:29:05.686423 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-396e5a5b-18c7-4f04-b043-36c1210037b4 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:29:05.687876 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-396e5a5b-18c7-4f04-b043-36c1210037b4 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:29:05.689226 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-396e5a5b-18c7-4f04-b043-36c1210037b4 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] 10.222.0.52 "GET /compute/v2.1/servers/56dbfa6a-5b6f-4b82-977b-af735e6a6ac9" status: 200 len: 1722 microversion: 2.20 time: 0.221978 Aug 27 02:29:05.689785 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 332/669] 10.222.0.52 () {60 vars in 1277 bytes} [Sat Aug 27 02:29:05 2022] GET /compute/v2.1/servers/56dbfa6a-5b6f-4b82-977b-af735e6a6ac9 => generated 1722 bytes in 223 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 27 02:29:06.723130 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-4e82872c-b848-4215-8b86-0d1bdc56da8d tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Calling method '>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:29:06.723761 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-4e82872c-b848-4215-8b86-0d1bdc56da8d tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] [instance: 56dbfa6a-5b6f-4b82-977b-af735e6a6ac9] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:29:06.731037 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-4e82872c-b848-4215-8b86-0d1bdc56da8d tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:29:06.731521 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-4e82872c-b848-4215-8b86-0d1bdc56da8d tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:29:06.731954 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-4e82872c-b848-4215-8b86-0d1bdc56da8d tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:29:06.732666 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-4e82872c-b848-4215-8b86-0d1bdc56da8d tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:29:06.733114 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-4e82872c-b848-4215-8b86-0d1bdc56da8d tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:29:06.733635 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-4e82872c-b848-4215-8b86-0d1bdc56da8d tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:29:06.876639 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-4e82872c-b848-4215-8b86-0d1bdc56da8d tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:29:06.886293 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-4e82872c-b848-4215-8b86-0d1bdc56da8d tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:29:06.887717 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-4e82872c-b848-4215-8b86-0d1bdc56da8d tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:29:06.888885 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-4e82872c-b848-4215-8b86-0d1bdc56da8d tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] 10.222.0.52 "GET /compute/v2.1/servers/56dbfa6a-5b6f-4b82-977b-af735e6a6ac9" status: 200 len: 1722 microversion: 2.20 time: 0.169897 Aug 27 02:29:06.889486 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 338/670] 10.222.0.52 () {60 vars in 1277 bytes} [Sat Aug 27 02:29:06 2022] GET /compute/v2.1/servers/56dbfa6a-5b6f-4b82-977b-af735e6a6ac9 => generated 1722 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Aug 27 02:29:07.588182 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [req-e281a15e-a70d-49c6-9e2e-2265b962559a req-40ea26ae-0fec-4737-b6e2-be4e942eded1 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "56dbfa6a-5b6f-4b82-977b-af735e6a6ac9", "name": "network-vif-unplugged", "status": "completed", "tag": "520bd5fb-723a-4570-83ca-72759c588707"}]} {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 27 02:29:07.600877 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [req-e281a15e-a70d-49c6-9e2e-2265b962559a req-40ea26ae-0fec-4737-b6e2-be4e942eded1 service nova] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:29:07.601728 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [req-e281a15e-a70d-49c6-9e2e-2265b962559a req-40ea26ae-0fec-4737-b6e2-be4e942eded1 service nova] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:29:07.602259 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [req-e281a15e-a70d-49c6-9e2e-2265b962559a req-40ea26ae-0fec-4737-b6e2-be4e942eded1 service nova] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:29:07.615543 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.compute.server_external_events [req-e281a15e-a70d-49c6-9e2e-2265b962559a req-40ea26ae-0fec-4737-b6e2-be4e942eded1 service nova] Creating event network-vif-unplugged:520bd5fb-723a-4570-83ca-72759c588707 for instance 56dbfa6a-5b6f-4b82-977b-af735e6a6ac9 on hv-cinder-84630 Aug 27 02:29:07.620434 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [req-e281a15e-a70d-49c6-9e2e-2265b962559a req-40ea26ae-0fec-4737-b6e2-be4e942eded1 service nova] 10.222.0.52 "POST /compute/v2.1/os-server-external-events" status: 200 len: 185 microversion: 2.1 time: 0.036204 Aug 27 02:29:07.621345 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 333/671] 10.222.0.52 () {64 vars in 1317 bytes} [Sat Aug 27 02:29:07 2022] POST /compute/v2.1/os-server-external-events => generated 185 bytes in 38 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 27 02:29:07.913514 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-76773b86-f2b8-4e28-aeee-1bd063a737b3 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Calling method '>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:29:07.914308 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-76773b86-f2b8-4e28-aeee-1bd063a737b3 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] [instance: 56dbfa6a-5b6f-4b82-977b-af735e6a6ac9] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:29:07.922023 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-76773b86-f2b8-4e28-aeee-1bd063a737b3 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:29:07.922505 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-76773b86-f2b8-4e28-aeee-1bd063a737b3 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:29:07.922899 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-76773b86-f2b8-4e28-aeee-1bd063a737b3 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:29:07.923589 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-76773b86-f2b8-4e28-aeee-1bd063a737b3 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:29:07.924014 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-76773b86-f2b8-4e28-aeee-1bd063a737b3 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:29:07.924429 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-76773b86-f2b8-4e28-aeee-1bd063a737b3 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:29:08.328870 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-76773b86-f2b8-4e28-aeee-1bd063a737b3 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:29:08.338189 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-76773b86-f2b8-4e28-aeee-1bd063a737b3 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:29:08.339792 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-76773b86-f2b8-4e28-aeee-1bd063a737b3 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:29:08.341415 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-76773b86-f2b8-4e28-aeee-1bd063a737b3 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] 10.222.0.52 "GET /compute/v2.1/servers/56dbfa6a-5b6f-4b82-977b-af735e6a6ac9" status: 200 len: 1725 microversion: 2.20 time: 0.433329 Aug 27 02:29:08.341984 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 339/672] 10.222.0.52 () {60 vars in 1277 bytes} [Sat Aug 27 02:29:07 2022] GET /compute/v2.1/servers/56dbfa6a-5b6f-4b82-977b-af735e6a6ac9 => generated 1725 bytes in 435 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 27 02:29:08.375180 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-f785cb0e-0b90-4165-9d14-286585a96eae tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "21b21458-c6cc-4e06-9522-5a468fb00a22"}} {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 27 02:29:08.377625 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-f785cb0e-0b90-4165-9d14-286585a96eae tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] [instance: 56dbfa6a-5b6f-4b82-977b-af735e6a6ac9] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:29:08.384224 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-f785cb0e-0b90-4165-9d14-286585a96eae tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:29:08.384708 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-f785cb0e-0b90-4165-9d14-286585a96eae tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:29:08.385158 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-f785cb0e-0b90-4165-9d14-286585a96eae tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:29:08.385860 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-f785cb0e-0b90-4165-9d14-286585a96eae tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:29:08.386332 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-f785cb0e-0b90-4165-9d14-286585a96eae tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:29:08.386741 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-f785cb0e-0b90-4165-9d14-286585a96eae tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:29:08.717765 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-f785cb0e-0b90-4165-9d14-286585a96eae tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] 10.222.0.52 "POST /compute/v2.1/servers/56dbfa6a-5b6f-4b82-977b-af735e6a6ac9/os-volume_attachments" status: 200 len: 188 microversion: 2.20 time: 0.349193 Aug 27 02:29:08.718667 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 334/673] 10.222.0.52 () {62 vars in 1364 bytes} [Sat Aug 27 02:29:08 2022] POST /compute/v2.1/servers/56dbfa6a-5b6f-4b82-977b-af735e6a6ac9/os-volume_attachments => generated 188 bytes in 351 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Aug 27 02:29:08.861169 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-be0717f9-010c-4ade-bb5e-d26705f56183 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Action: 'action', calling method: >, body: {"unshelve": null} {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 27 02:29:08.861779 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-be0717f9-010c-4ade-bb5e-d26705f56183 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] [instance: 56dbfa6a-5b6f-4b82-977b-af735e6a6ac9] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:29:08.868435 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-be0717f9-010c-4ade-bb5e-d26705f56183 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:29:08.868917 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-be0717f9-010c-4ade-bb5e-d26705f56183 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:29:08.869413 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-be0717f9-010c-4ade-bb5e-d26705f56183 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:29:08.870117 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-be0717f9-010c-4ade-bb5e-d26705f56183 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:29:08.870598 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-be0717f9-010c-4ade-bb5e-d26705f56183 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:29:08.871015 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-be0717f9-010c-4ade-bb5e-d26705f56183 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:29:09.085897 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-be0717f9-010c-4ade-bb5e-d26705f56183 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] [instance: 56dbfa6a-5b6f-4b82-977b-af735e6a6ac9] Unshelving instance with old availability_zone "None" to new availability_zone "not provided" and host "None". {{(pid=120864) unshelve /opt/stack/nova/nova/compute/api.py:4591}} Aug 27 02:29:09.171228 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-be0717f9-010c-4ade-bb5e-d26705f56183 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] 10.222.0.52 "POST /compute/v2.1/servers/56dbfa6a-5b6f-4b82-977b-af735e6a6ac9/action" status: 202 len: 0 microversion: 2.20 time: 0.314109 Aug 27 02:29:09.175451 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 340/674] 10.222.0.52 () {62 vars in 1319 bytes} [Sat Aug 27 02:29:08 2022] POST /compute/v2.1/servers/56dbfa6a-5b6f-4b82-977b-af735e6a6ac9/action => generated 0 bytes in 316 msecs (HTTP/1.1 202) 9 headers in 363 bytes (1 switches on core 0) Aug 27 02:29:09.183009 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-8102d596-cb67-4608-a69c-513a447eb4ab tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:29:09.183688 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-8102d596-cb67-4608-a69c-513a447eb4ab tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] [instance: 56dbfa6a-5b6f-4b82-977b-af735e6a6ac9] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:29:09.191209 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-8102d596-cb67-4608-a69c-513a447eb4ab tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:29:09.191209 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-8102d596-cb67-4608-a69c-513a447eb4ab tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:29:09.191587 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-8102d596-cb67-4608-a69c-513a447eb4ab tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:29:09.192297 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-8102d596-cb67-4608-a69c-513a447eb4ab tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:29:09.192767 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-8102d596-cb67-4608-a69c-513a447eb4ab tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:29:09.193287 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-8102d596-cb67-4608-a69c-513a447eb4ab tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:29:09.347470 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-8102d596-cb67-4608-a69c-513a447eb4ab tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:29:09.359888 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-8102d596-cb67-4608-a69c-513a447eb4ab tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:29:09.361504 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-8102d596-cb67-4608-a69c-513a447eb4ab tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:29:09.362803 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-8102d596-cb67-4608-a69c-513a447eb4ab tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] 10.222.0.52 "GET /compute/v2.1/servers/56dbfa6a-5b6f-4b82-977b-af735e6a6ac9" status: 200 len: 1751 microversion: 2.20 time: 0.185658 Aug 27 02:29:09.363377 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 335/675] 10.222.0.52 () {60 vars in 1277 bytes} [Sat Aug 27 02:29:09 2022] GET /compute/v2.1/servers/56dbfa6a-5b6f-4b82-977b-af735e6a6ac9 => generated 1751 bytes in 187 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 27 02:29:10.406007 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-fc9d84e7-cdab-4468-9f07-3572d03d9f16 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Calling method '>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:29:10.406930 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-fc9d84e7-cdab-4468-9f07-3572d03d9f16 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] [instance: 56dbfa6a-5b6f-4b82-977b-af735e6a6ac9] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:29:10.415324 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-fc9d84e7-cdab-4468-9f07-3572d03d9f16 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:29:10.415884 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-fc9d84e7-cdab-4468-9f07-3572d03d9f16 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:29:10.416538 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-fc9d84e7-cdab-4468-9f07-3572d03d9f16 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:29:10.417426 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-fc9d84e7-cdab-4468-9f07-3572d03d9f16 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:29:10.418041 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-fc9d84e7-cdab-4468-9f07-3572d03d9f16 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:29:10.418548 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-fc9d84e7-cdab-4468-9f07-3572d03d9f16 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:29:10.575960 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-fc9d84e7-cdab-4468-9f07-3572d03d9f16 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:29:10.585588 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-fc9d84e7-cdab-4468-9f07-3572d03d9f16 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:29:10.587052 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-fc9d84e7-cdab-4468-9f07-3572d03d9f16 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:29:10.588585 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-fc9d84e7-cdab-4468-9f07-3572d03d9f16 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] 10.222.0.52 "GET /compute/v2.1/servers/56dbfa6a-5b6f-4b82-977b-af735e6a6ac9" status: 200 len: 1753 microversion: 2.20 time: 0.194156 Aug 27 02:29:10.589108 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 341/676] 10.222.0.52 () {60 vars in 1277 bytes} [Sat Aug 27 02:29:10 2022] GET /compute/v2.1/servers/56dbfa6a-5b6f-4b82-977b-af735e6a6ac9 => generated 1753 bytes in 195 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 27 02:29:11.614663 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-cebcd4dd-38a4-43bb-ba2d-8c7d605a969f tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:29:11.615676 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-cebcd4dd-38a4-43bb-ba2d-8c7d605a969f tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] [instance: 56dbfa6a-5b6f-4b82-977b-af735e6a6ac9] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:29:11.626029 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-cebcd4dd-38a4-43bb-ba2d-8c7d605a969f tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:29:11.626917 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-cebcd4dd-38a4-43bb-ba2d-8c7d605a969f tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:29:11.627545 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-cebcd4dd-38a4-43bb-ba2d-8c7d605a969f tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:29:11.628264 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-cebcd4dd-38a4-43bb-ba2d-8c7d605a969f tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:29:11.628722 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-cebcd4dd-38a4-43bb-ba2d-8c7d605a969f tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:29:11.629168 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-cebcd4dd-38a4-43bb-ba2d-8c7d605a969f tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:29:11.804551 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-cebcd4dd-38a4-43bb-ba2d-8c7d605a969f tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:29:11.813981 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-cebcd4dd-38a4-43bb-ba2d-8c7d605a969f tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:29:11.815697 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-cebcd4dd-38a4-43bb-ba2d-8c7d605a969f tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:29:11.817160 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-cebcd4dd-38a4-43bb-ba2d-8c7d605a969f tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] 10.222.0.52 "GET /compute/v2.1/servers/56dbfa6a-5b6f-4b82-977b-af735e6a6ac9" status: 200 len: 1809 microversion: 2.20 time: 0.206938 Aug 27 02:29:11.817836 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 336/677] 10.222.0.52 () {60 vars in 1277 bytes} [Sat Aug 27 02:29:11 2022] GET /compute/v2.1/servers/56dbfa6a-5b6f-4b82-977b-af735e6a6ac9 => generated 1809 bytes in 209 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 27 02:29:12.841555 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-7a997f5a-b0f7-4658-8815-36620d0ac9e8 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Calling method '>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:29:12.842294 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-7a997f5a-b0f7-4658-8815-36620d0ac9e8 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] [instance: 56dbfa6a-5b6f-4b82-977b-af735e6a6ac9] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:29:12.851647 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-7a997f5a-b0f7-4658-8815-36620d0ac9e8 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:29:12.852142 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-7a997f5a-b0f7-4658-8815-36620d0ac9e8 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:29:12.852580 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-7a997f5a-b0f7-4658-8815-36620d0ac9e8 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:29:12.853323 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-7a997f5a-b0f7-4658-8815-36620d0ac9e8 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:29:12.853774 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-7a997f5a-b0f7-4658-8815-36620d0ac9e8 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:29:12.854567 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-7a997f5a-b0f7-4658-8815-36620d0ac9e8 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:29:13.008014 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-7a997f5a-b0f7-4658-8815-36620d0ac9e8 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:29:13.018686 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-7a997f5a-b0f7-4658-8815-36620d0ac9e8 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:29:13.020234 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-7a997f5a-b0f7-4658-8815-36620d0ac9e8 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:29:13.021458 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-7a997f5a-b0f7-4658-8815-36620d0ac9e8 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] 10.222.0.52 "GET /compute/v2.1/servers/56dbfa6a-5b6f-4b82-977b-af735e6a6ac9" status: 200 len: 1809 microversion: 2.20 time: 0.184195 Aug 27 02:29:13.022038 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 342/678] 10.222.0.52 () {60 vars in 1277 bytes} [Sat Aug 27 02:29:12 2022] GET /compute/v2.1/servers/56dbfa6a-5b6f-4b82-977b-af735e6a6ac9 => generated 1809 bytes in 185 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 27 02:29:14.048288 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-0ea8ff38-a1bd-4147-9956-836f9d670110 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:29:14.048879 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-0ea8ff38-a1bd-4147-9956-836f9d670110 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] [instance: 56dbfa6a-5b6f-4b82-977b-af735e6a6ac9] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:29:14.051413 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG dbcounter [-] [120863] Writing DB stats nova_api:SELECT=28 {{(pid=120863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 27 02:29:14.058302 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-0ea8ff38-a1bd-4147-9956-836f9d670110 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:29:14.058796 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-0ea8ff38-a1bd-4147-9956-836f9d670110 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:29:14.059383 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-0ea8ff38-a1bd-4147-9956-836f9d670110 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:29:14.060189 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-0ea8ff38-a1bd-4147-9956-836f9d670110 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:29:14.060705 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-0ea8ff38-a1bd-4147-9956-836f9d670110 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:29:14.061175 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-0ea8ff38-a1bd-4147-9956-836f9d670110 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:29:14.258826 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-0ea8ff38-a1bd-4147-9956-836f9d670110 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:29:14.273934 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-0ea8ff38-a1bd-4147-9956-836f9d670110 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:29:14.276091 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-0ea8ff38-a1bd-4147-9956-836f9d670110 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:29:14.277508 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-0ea8ff38-a1bd-4147-9956-836f9d670110 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] 10.222.0.52 "GET /compute/v2.1/servers/56dbfa6a-5b6f-4b82-977b-af735e6a6ac9" status: 200 len: 1809 microversion: 2.20 time: 0.233430 Aug 27 02:29:14.278218 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 337/679] 10.222.0.52 () {60 vars in 1277 bytes} [Sat Aug 27 02:29:14 2022] GET /compute/v2.1/servers/56dbfa6a-5b6f-4b82-977b-af735e6a6ac9 => generated 1809 bytes in 235 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 27 02:29:15.301546 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-f8e27271-e6cc-44a8-b6fb-311955925e51 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Calling method '>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:29:15.302145 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-f8e27271-e6cc-44a8-b6fb-311955925e51 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] [instance: 56dbfa6a-5b6f-4b82-977b-af735e6a6ac9] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:29:15.304707 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG dbcounter [-] [120864] Writing DB stats nova_api:SELECT=32,nova_api:UPDATE=1 {{(pid=120864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 27 02:29:15.310436 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-f8e27271-e6cc-44a8-b6fb-311955925e51 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:29:15.310981 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-f8e27271-e6cc-44a8-b6fb-311955925e51 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:29:15.311418 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-f8e27271-e6cc-44a8-b6fb-311955925e51 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:29:15.312115 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-f8e27271-e6cc-44a8-b6fb-311955925e51 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:29:15.312569 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-f8e27271-e6cc-44a8-b6fb-311955925e51 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:29:15.312983 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-f8e27271-e6cc-44a8-b6fb-311955925e51 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:29:15.465212 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-f8e27271-e6cc-44a8-b6fb-311955925e51 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:29:15.474259 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-f8e27271-e6cc-44a8-b6fb-311955925e51 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:29:15.475691 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-f8e27271-e6cc-44a8-b6fb-311955925e51 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:29:15.476903 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-f8e27271-e6cc-44a8-b6fb-311955925e51 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] 10.222.0.52 "GET /compute/v2.1/servers/56dbfa6a-5b6f-4b82-977b-af735e6a6ac9" status: 200 len: 1809 microversion: 2.20 time: 0.179258 Aug 27 02:29:15.477586 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 343/680] 10.222.0.52 () {60 vars in 1277 bytes} [Sat Aug 27 02:29:15 2022] GET /compute/v2.1/servers/56dbfa6a-5b6f-4b82-977b-af735e6a6ac9 => generated 1809 bytes in 181 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 27 02:29:16.504330 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-344b5dd5-db34-47af-adbc-69a64e8f241c tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:29:16.505171 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-344b5dd5-db34-47af-adbc-69a64e8f241c tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] [instance: 56dbfa6a-5b6f-4b82-977b-af735e6a6ac9] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:29:16.511736 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-344b5dd5-db34-47af-adbc-69a64e8f241c tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:29:16.512466 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-344b5dd5-db34-47af-adbc-69a64e8f241c tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:29:16.512911 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-344b5dd5-db34-47af-adbc-69a64e8f241c tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:29:16.513643 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-344b5dd5-db34-47af-adbc-69a64e8f241c tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:29:16.514097 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-344b5dd5-db34-47af-adbc-69a64e8f241c tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:29:16.514520 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-344b5dd5-db34-47af-adbc-69a64e8f241c tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:29:16.663755 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-344b5dd5-db34-47af-adbc-69a64e8f241c tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:29:16.678843 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-344b5dd5-db34-47af-adbc-69a64e8f241c tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:29:16.681804 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-344b5dd5-db34-47af-adbc-69a64e8f241c tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:29:16.683680 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-344b5dd5-db34-47af-adbc-69a64e8f241c tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] 10.222.0.52 "GET /compute/v2.1/servers/56dbfa6a-5b6f-4b82-977b-af735e6a6ac9" status: 200 len: 1809 microversion: 2.20 time: 0.183891 Aug 27 02:29:16.684260 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 338/681] 10.222.0.52 () {60 vars in 1277 bytes} [Sat Aug 27 02:29:16 2022] GET /compute/v2.1/servers/56dbfa6a-5b6f-4b82-977b-af735e6a6ac9 => generated 1809 bytes in 185 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 27 02:29:17.707526 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-50bca15f-b4a5-4afd-8961-1efb16f31f15 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Calling method '>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:29:17.708140 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-50bca15f-b4a5-4afd-8961-1efb16f31f15 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] [instance: 56dbfa6a-5b6f-4b82-977b-af735e6a6ac9] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:29:17.716161 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-50bca15f-b4a5-4afd-8961-1efb16f31f15 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:29:17.716630 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-50bca15f-b4a5-4afd-8961-1efb16f31f15 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:29:17.717098 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-50bca15f-b4a5-4afd-8961-1efb16f31f15 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:29:17.718000 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-50bca15f-b4a5-4afd-8961-1efb16f31f15 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:29:17.718512 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-50bca15f-b4a5-4afd-8961-1efb16f31f15 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:29:17.718939 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-50bca15f-b4a5-4afd-8961-1efb16f31f15 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:29:17.889723 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-50bca15f-b4a5-4afd-8961-1efb16f31f15 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:29:17.900464 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-50bca15f-b4a5-4afd-8961-1efb16f31f15 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:29:17.901922 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-50bca15f-b4a5-4afd-8961-1efb16f31f15 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:29:17.903288 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-50bca15f-b4a5-4afd-8961-1efb16f31f15 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] 10.222.0.52 "GET /compute/v2.1/servers/56dbfa6a-5b6f-4b82-977b-af735e6a6ac9" status: 200 len: 1809 microversion: 2.20 time: 0.199679 Aug 27 02:29:17.904105 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 344/682] 10.222.0.52 () {60 vars in 1277 bytes} [Sat Aug 27 02:29:17 2022] GET /compute/v2.1/servers/56dbfa6a-5b6f-4b82-977b-af735e6a6ac9 => generated 1809 bytes in 201 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 27 02:29:18.928325 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-462a1ae3-feaa-4b14-9e58-22cc721f03a7 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:29:18.929024 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-462a1ae3-feaa-4b14-9e58-22cc721f03a7 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] [instance: 56dbfa6a-5b6f-4b82-977b-af735e6a6ac9] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:29:18.936574 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-462a1ae3-feaa-4b14-9e58-22cc721f03a7 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:29:18.937067 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-462a1ae3-feaa-4b14-9e58-22cc721f03a7 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:29:18.937533 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-462a1ae3-feaa-4b14-9e58-22cc721f03a7 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:29:18.938253 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-462a1ae3-feaa-4b14-9e58-22cc721f03a7 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:29:18.938710 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-462a1ae3-feaa-4b14-9e58-22cc721f03a7 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:29:18.939134 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-462a1ae3-feaa-4b14-9e58-22cc721f03a7 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:29:19.115114 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-462a1ae3-feaa-4b14-9e58-22cc721f03a7 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:29:19.144902 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-462a1ae3-feaa-4b14-9e58-22cc721f03a7 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:29:19.146571 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-462a1ae3-feaa-4b14-9e58-22cc721f03a7 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:29:19.148468 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-462a1ae3-feaa-4b14-9e58-22cc721f03a7 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] 10.222.0.52 "GET /compute/v2.1/servers/56dbfa6a-5b6f-4b82-977b-af735e6a6ac9" status: 200 len: 1809 microversion: 2.20 time: 0.224082 Aug 27 02:29:19.166551 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 339/683] 10.222.0.52 () {60 vars in 1277 bytes} [Sat Aug 27 02:29:18 2022] GET /compute/v2.1/servers/56dbfa6a-5b6f-4b82-977b-af735e6a6ac9 => generated 1809 bytes in 242 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 27 02:29:20.176710 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-9ace5921-0d67-4ebd-8278-6df344538dca tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Calling method '>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:29:20.177377 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-9ace5921-0d67-4ebd-8278-6df344538dca tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] [instance: 56dbfa6a-5b6f-4b82-977b-af735e6a6ac9] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:29:20.188029 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-9ace5921-0d67-4ebd-8278-6df344538dca tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:29:20.188630 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-9ace5921-0d67-4ebd-8278-6df344538dca tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:29:20.189142 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-9ace5921-0d67-4ebd-8278-6df344538dca tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:29:20.189905 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-9ace5921-0d67-4ebd-8278-6df344538dca tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:29:20.190415 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-9ace5921-0d67-4ebd-8278-6df344538dca tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:29:20.190831 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-9ace5921-0d67-4ebd-8278-6df344538dca tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:29:20.216672 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG dbcounter [-] [120864] Writing DB stats nova_cell1:SELECT=88,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=120864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 27 02:29:20.420138 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-9ace5921-0d67-4ebd-8278-6df344538dca tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:29:20.431079 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-9ace5921-0d67-4ebd-8278-6df344538dca tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:29:20.432649 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-9ace5921-0d67-4ebd-8278-6df344538dca tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:29:20.433931 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-9ace5921-0d67-4ebd-8278-6df344538dca tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] 10.222.0.52 "GET /compute/v2.1/servers/56dbfa6a-5b6f-4b82-977b-af735e6a6ac9" status: 200 len: 1809 microversion: 2.20 time: 0.262511 Aug 27 02:29:20.434530 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 345/684] 10.222.0.52 () {60 vars in 1277 bytes} [Sat Aug 27 02:29:20 2022] GET /compute/v2.1/servers/56dbfa6a-5b6f-4b82-977b-af735e6a6ac9 => generated 1809 bytes in 264 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 27 02:29:21.460135 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-e3e1d565-42f8-4928-aadf-6371c638ebcf tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:29:21.461694 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-e3e1d565-42f8-4928-aadf-6371c638ebcf tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] [instance: 56dbfa6a-5b6f-4b82-977b-af735e6a6ac9] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:29:21.469498 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-e3e1d565-42f8-4928-aadf-6371c638ebcf tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:29:21.469909 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-e3e1d565-42f8-4928-aadf-6371c638ebcf tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:29:21.470494 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-e3e1d565-42f8-4928-aadf-6371c638ebcf tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:29:21.471256 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-e3e1d565-42f8-4928-aadf-6371c638ebcf tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:29:21.471782 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-e3e1d565-42f8-4928-aadf-6371c638ebcf tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:29:21.472267 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-e3e1d565-42f8-4928-aadf-6371c638ebcf tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:29:21.650865 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-e3e1d565-42f8-4928-aadf-6371c638ebcf tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:29:21.663709 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-e3e1d565-42f8-4928-aadf-6371c638ebcf tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:29:21.665870 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-e3e1d565-42f8-4928-aadf-6371c638ebcf tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:29:21.667065 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-e3e1d565-42f8-4928-aadf-6371c638ebcf tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] 10.222.0.52 "GET /compute/v2.1/servers/56dbfa6a-5b6f-4b82-977b-af735e6a6ac9" status: 200 len: 1809 microversion: 2.20 time: 0.212976 Aug 27 02:29:21.667638 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 340/685] 10.222.0.52 () {60 vars in 1277 bytes} [Sat Aug 27 02:29:21 2022] GET /compute/v2.1/servers/56dbfa6a-5b6f-4b82-977b-af735e6a6ac9 => generated 1809 bytes in 214 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 27 02:29:22.692995 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-ffa12b3c-162a-4752-8f3f-15267676eeec tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Calling method '>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:29:22.694461 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-ffa12b3c-162a-4752-8f3f-15267676eeec tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] [instance: 56dbfa6a-5b6f-4b82-977b-af735e6a6ac9] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:29:22.705502 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-ffa12b3c-162a-4752-8f3f-15267676eeec tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:29:22.706033 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-ffa12b3c-162a-4752-8f3f-15267676eeec tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:29:22.706544 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-ffa12b3c-162a-4752-8f3f-15267676eeec tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:29:22.707440 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-ffa12b3c-162a-4752-8f3f-15267676eeec tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:29:22.707829 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-ffa12b3c-162a-4752-8f3f-15267676eeec tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:29:22.708274 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-ffa12b3c-162a-4752-8f3f-15267676eeec tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:29:22.872638 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-ffa12b3c-162a-4752-8f3f-15267676eeec tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:29:22.885427 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-ffa12b3c-162a-4752-8f3f-15267676eeec tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:29:22.888490 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-ffa12b3c-162a-4752-8f3f-15267676eeec tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:29:22.890477 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-ffa12b3c-162a-4752-8f3f-15267676eeec tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] 10.222.0.52 "GET /compute/v2.1/servers/56dbfa6a-5b6f-4b82-977b-af735e6a6ac9" status: 200 len: 1809 microversion: 2.20 time: 0.203757 Aug 27 02:29:22.898976 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 346/686] 10.222.0.52 () {60 vars in 1277 bytes} [Sat Aug 27 02:29:22 2022] GET /compute/v2.1/servers/56dbfa6a-5b6f-4b82-977b-af735e6a6ac9 => generated 1809 bytes in 206 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Aug 27 02:29:23.923968 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-ca685ae9-a80d-4d5e-9606-638e0830f568 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:29:23.924671 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-ca685ae9-a80d-4d5e-9606-638e0830f568 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] [instance: 56dbfa6a-5b6f-4b82-977b-af735e6a6ac9] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:29:23.931641 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-ca685ae9-a80d-4d5e-9606-638e0830f568 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:29:23.932126 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-ca685ae9-a80d-4d5e-9606-638e0830f568 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:29:23.932554 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-ca685ae9-a80d-4d5e-9606-638e0830f568 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:29:23.933290 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-ca685ae9-a80d-4d5e-9606-638e0830f568 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:29:23.933748 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-ca685ae9-a80d-4d5e-9606-638e0830f568 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:29:23.934182 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-ca685ae9-a80d-4d5e-9606-638e0830f568 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:29:23.936894 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG dbcounter [-] [120863] Writing DB stats nova_cell1:SELECT=91,nova_cell1:INSERT=3,nova_cell1:UPDATE=4 {{(pid=120863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 27 02:29:24.095643 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-ca685ae9-a80d-4d5e-9606-638e0830f568 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:29:24.106176 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-ca685ae9-a80d-4d5e-9606-638e0830f568 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:29:24.107207 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-ca685ae9-a80d-4d5e-9606-638e0830f568 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:29:24.108475 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-ca685ae9-a80d-4d5e-9606-638e0830f568 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] 10.222.0.52 "GET /compute/v2.1/servers/56dbfa6a-5b6f-4b82-977b-af735e6a6ac9" status: 200 len: 1809 microversion: 2.20 time: 0.188613 Aug 27 02:29:24.109056 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 341/687] 10.222.0.52 () {60 vars in 1277 bytes} [Sat Aug 27 02:29:23 2022] GET /compute/v2.1/servers/56dbfa6a-5b6f-4b82-977b-af735e6a6ac9 => generated 1809 bytes in 190 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 27 02:29:25.132851 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-803fa180-e30a-41ed-97eb-ebc219d2f38a tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Calling method '>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:29:25.133846 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-803fa180-e30a-41ed-97eb-ebc219d2f38a tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] [instance: 56dbfa6a-5b6f-4b82-977b-af735e6a6ac9] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:29:25.141706 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-803fa180-e30a-41ed-97eb-ebc219d2f38a tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:29:25.142188 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-803fa180-e30a-41ed-97eb-ebc219d2f38a tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:29:25.142608 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-803fa180-e30a-41ed-97eb-ebc219d2f38a tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:29:25.143537 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-803fa180-e30a-41ed-97eb-ebc219d2f38a tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:29:25.144109 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-803fa180-e30a-41ed-97eb-ebc219d2f38a tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:29:25.144743 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-803fa180-e30a-41ed-97eb-ebc219d2f38a tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:29:25.294934 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-803fa180-e30a-41ed-97eb-ebc219d2f38a tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:29:25.304033 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-803fa180-e30a-41ed-97eb-ebc219d2f38a tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:29:25.306271 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-803fa180-e30a-41ed-97eb-ebc219d2f38a tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:29:25.307546 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-803fa180-e30a-41ed-97eb-ebc219d2f38a tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] 10.222.0.52 "GET /compute/v2.1/servers/56dbfa6a-5b6f-4b82-977b-af735e6a6ac9" status: 200 len: 1809 microversion: 2.20 time: 0.179348 Aug 27 02:29:25.308110 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 347/688] 10.222.0.52 () {60 vars in 1277 bytes} [Sat Aug 27 02:29:25 2022] GET /compute/v2.1/servers/56dbfa6a-5b6f-4b82-977b-af735e6a6ac9 => generated 1809 bytes in 181 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 27 02:29:26.331788 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-8c1683c7-c7a5-41d5-8bc4-f2af9336b81a tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:29:26.332669 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-8c1683c7-c7a5-41d5-8bc4-f2af9336b81a tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] [instance: 56dbfa6a-5b6f-4b82-977b-af735e6a6ac9] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:29:26.340510 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-8c1683c7-c7a5-41d5-8bc4-f2af9336b81a tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:29:26.341013 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-8c1683c7-c7a5-41d5-8bc4-f2af9336b81a tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:29:26.341496 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-8c1683c7-c7a5-41d5-8bc4-f2af9336b81a tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:29:26.342370 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-8c1683c7-c7a5-41d5-8bc4-f2af9336b81a tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:29:26.342824 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-8c1683c7-c7a5-41d5-8bc4-f2af9336b81a tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:29:26.343255 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-8c1683c7-c7a5-41d5-8bc4-f2af9336b81a tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:29:26.482123 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-8c1683c7-c7a5-41d5-8bc4-f2af9336b81a tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:29:26.491375 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-8c1683c7-c7a5-41d5-8bc4-f2af9336b81a tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:29:26.492837 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-8c1683c7-c7a5-41d5-8bc4-f2af9336b81a tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:29:26.494027 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-8c1683c7-c7a5-41d5-8bc4-f2af9336b81a tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] 10.222.0.52 "GET /compute/v2.1/servers/56dbfa6a-5b6f-4b82-977b-af735e6a6ac9" status: 200 len: 1809 microversion: 2.20 time: 0.166291 Aug 27 02:29:26.494588 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 342/689] 10.222.0.52 () {60 vars in 1277 bytes} [Sat Aug 27 02:29:26 2022] GET /compute/v2.1/servers/56dbfa6a-5b6f-4b82-977b-af735e6a6ac9 => generated 1809 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 27 02:29:27.517695 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-c050c866-0442-4e46-8635-9b99e8fcd746 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:29:27.518404 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-c050c866-0442-4e46-8635-9b99e8fcd746 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] [instance: 56dbfa6a-5b6f-4b82-977b-af735e6a6ac9] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:29:27.528118 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-c050c866-0442-4e46-8635-9b99e8fcd746 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:29:27.528654 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-c050c866-0442-4e46-8635-9b99e8fcd746 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:29:27.529196 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-c050c866-0442-4e46-8635-9b99e8fcd746 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:29:27.529878 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-9d6e8e5b-6841-4f26-ae2e-e2bdce0fce36 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "2a8726a0-0c8d-4001-aaca-d11c877d0d88"}} {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 27 02:29:27.530382 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-c050c866-0442-4e46-8635-9b99e8fcd746 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:29:27.530486 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-c050c866-0442-4e46-8635-9b99e8fcd746 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:29:27.530994 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-c050c866-0442-4e46-8635-9b99e8fcd746 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:29:27.533023 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-9d6e8e5b-6841-4f26-ae2e-e2bdce0fce36 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] [instance: 98e4cfeb-d49e-4888-a2c0-40a9e8af90d3] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:29:27.539827 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-9d6e8e5b-6841-4f26-ae2e-e2bdce0fce36 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:29:27.540314 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-9d6e8e5b-6841-4f26-ae2e-e2bdce0fce36 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:29:27.540750 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-9d6e8e5b-6841-4f26-ae2e-e2bdce0fce36 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:29:27.541717 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-9d6e8e5b-6841-4f26-ae2e-e2bdce0fce36 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:29:27.542348 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-9d6e8e5b-6841-4f26-ae2e-e2bdce0fce36 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:29:27.542499 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-9d6e8e5b-6841-4f26-ae2e-e2bdce0fce36 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:29:27.697709 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-c050c866-0442-4e46-8635-9b99e8fcd746 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:29:27.706834 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-c050c866-0442-4e46-8635-9b99e8fcd746 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:29:27.708258 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-c050c866-0442-4e46-8635-9b99e8fcd746 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:29:27.709762 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-c050c866-0442-4e46-8635-9b99e8fcd746 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] 10.222.0.52 "GET /compute/v2.1/servers/56dbfa6a-5b6f-4b82-977b-af735e6a6ac9" status: 200 len: 1809 microversion: 2.20 time: 0.195981 Aug 27 02:29:27.710302 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 343/690] 10.222.0.52 () {60 vars in 1277 bytes} [Sat Aug 27 02:29:27 2022] GET /compute/v2.1/servers/56dbfa6a-5b6f-4b82-977b-af735e6a6ac9 => generated 1809 bytes in 197 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Aug 27 02:29:28.732798 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-ca74db6a-303c-4188-8caa-ebc56c50682d tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:29:28.733729 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-ca74db6a-303c-4188-8caa-ebc56c50682d tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] [instance: 56dbfa6a-5b6f-4b82-977b-af735e6a6ac9] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:29:28.740476 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-ca74db6a-303c-4188-8caa-ebc56c50682d tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:29:28.740968 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-ca74db6a-303c-4188-8caa-ebc56c50682d tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:29:28.741496 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-ca74db6a-303c-4188-8caa-ebc56c50682d tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:29:28.742330 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-ca74db6a-303c-4188-8caa-ebc56c50682d tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:29:28.742780 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-ca74db6a-303c-4188-8caa-ebc56c50682d tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:29:28.743198 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-ca74db6a-303c-4188-8caa-ebc56c50682d tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:29:28.888194 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-ca74db6a-303c-4188-8caa-ebc56c50682d tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:29:28.897326 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-ca74db6a-303c-4188-8caa-ebc56c50682d tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:29:28.899419 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-ca74db6a-303c-4188-8caa-ebc56c50682d tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:29:28.900598 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-ca74db6a-303c-4188-8caa-ebc56c50682d tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] 10.222.0.52 "GET /compute/v2.1/servers/56dbfa6a-5b6f-4b82-977b-af735e6a6ac9" status: 200 len: 1809 microversion: 2.20 time: 0.172352 Aug 27 02:29:28.901175 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 344/691] 10.222.0.52 () {60 vars in 1277 bytes} [Sat Aug 27 02:29:28 2022] GET /compute/v2.1/servers/56dbfa6a-5b6f-4b82-977b-af735e6a6ac9 => generated 1809 bytes in 174 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Aug 27 02:29:29.099040 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-9d6e8e5b-6841-4f26-ae2e-e2bdce0fce36 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] 10.222.0.52 "POST /compute/v2.1/servers/98e4cfeb-d49e-4888-a2c0-40a9e8af90d3/os-volume_attachments" status: 200 len: 194 microversion: 2.1 time: 1.692638 Aug 27 02:29:29.099698 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 348/692] 10.222.0.52 () {60 vars in 1323 bytes} [Sat Aug 27 02:29:27 2022] POST /compute/v2.1/servers/98e4cfeb-d49e-4888-a2c0-40a9e8af90d3/os-volume_attachments => generated 194 bytes in 1694 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 27 02:29:29.923416 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-4b90ece6-1ef3-41fd-bf22-d7eec462ecef tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:29:29.924198 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-4b90ece6-1ef3-41fd-bf22-d7eec462ecef tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] [instance: 56dbfa6a-5b6f-4b82-977b-af735e6a6ac9] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:29:29.931202 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-4b90ece6-1ef3-41fd-bf22-d7eec462ecef tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:29:29.931684 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-4b90ece6-1ef3-41fd-bf22-d7eec462ecef tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:29:29.932106 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-4b90ece6-1ef3-41fd-bf22-d7eec462ecef tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:29:29.932790 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-4b90ece6-1ef3-41fd-bf22-d7eec462ecef tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:29:29.933364 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-4b90ece6-1ef3-41fd-bf22-d7eec462ecef tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:29:29.933781 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-4b90ece6-1ef3-41fd-bf22-d7eec462ecef tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:29:30.100778 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-4b90ece6-1ef3-41fd-bf22-d7eec462ecef tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:29:30.113383 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-4b90ece6-1ef3-41fd-bf22-d7eec462ecef tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:29:30.115654 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-4b90ece6-1ef3-41fd-bf22-d7eec462ecef tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:29:30.116998 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-4b90ece6-1ef3-41fd-bf22-d7eec462ecef tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] 10.222.0.52 "GET /compute/v2.1/servers/56dbfa6a-5b6f-4b82-977b-af735e6a6ac9" status: 200 len: 1809 microversion: 2.20 time: 0.197651 Aug 27 02:29:30.117914 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 345/693] 10.222.0.52 () {60 vars in 1277 bytes} [Sat Aug 27 02:29:29 2022] GET /compute/v2.1/servers/56dbfa6a-5b6f-4b82-977b-af735e6a6ac9 => generated 1809 bytes in 199 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 27 02:29:31.152139 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-4836df63-7cab-4078-9147-c000722a5be3 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Calling method '>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:29:31.152835 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-4836df63-7cab-4078-9147-c000722a5be3 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] [instance: 56dbfa6a-5b6f-4b82-977b-af735e6a6ac9] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:29:31.159225 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-4836df63-7cab-4078-9147-c000722a5be3 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:29:31.159702 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-4836df63-7cab-4078-9147-c000722a5be3 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:29:31.160150 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-4836df63-7cab-4078-9147-c000722a5be3 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:29:31.160845 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-4836df63-7cab-4078-9147-c000722a5be3 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:29:31.161320 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-4836df63-7cab-4078-9147-c000722a5be3 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:29:31.161746 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-4836df63-7cab-4078-9147-c000722a5be3 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:29:31.313796 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-4836df63-7cab-4078-9147-c000722a5be3 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:29:31.323972 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-4836df63-7cab-4078-9147-c000722a5be3 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:29:31.325511 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-4836df63-7cab-4078-9147-c000722a5be3 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:29:31.326722 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-4836df63-7cab-4078-9147-c000722a5be3 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] 10.222.0.52 "GET /compute/v2.1/servers/56dbfa6a-5b6f-4b82-977b-af735e6a6ac9" status: 200 len: 1809 microversion: 2.20 time: 0.179769 Aug 27 02:29:31.327326 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 349/694] 10.222.0.52 () {60 vars in 1277 bytes} [Sat Aug 27 02:29:31 2022] GET /compute/v2.1/servers/56dbfa6a-5b6f-4b82-977b-af735e6a6ac9 => generated 1809 bytes in 181 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 27 02:29:32.353890 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-ec63f73f-2d8d-4b07-a50b-34ce4fdb4891 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:29:32.354855 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-ec63f73f-2d8d-4b07-a50b-34ce4fdb4891 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] [instance: 56dbfa6a-5b6f-4b82-977b-af735e6a6ac9] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:29:32.362499 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-ec63f73f-2d8d-4b07-a50b-34ce4fdb4891 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:29:32.363407 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-ec63f73f-2d8d-4b07-a50b-34ce4fdb4891 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:29:32.363407 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-ec63f73f-2d8d-4b07-a50b-34ce4fdb4891 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:29:32.364184 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-ec63f73f-2d8d-4b07-a50b-34ce4fdb4891 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:29:32.364738 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-ec63f73f-2d8d-4b07-a50b-34ce4fdb4891 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:29:32.365870 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-ec63f73f-2d8d-4b07-a50b-34ce4fdb4891 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:29:32.510242 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-ec63f73f-2d8d-4b07-a50b-34ce4fdb4891 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:29:32.519420 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-ec63f73f-2d8d-4b07-a50b-34ce4fdb4891 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:29:32.520858 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-ec63f73f-2d8d-4b07-a50b-34ce4fdb4891 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:29:32.522050 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-ec63f73f-2d8d-4b07-a50b-34ce4fdb4891 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] 10.222.0.52 "GET /compute/v2.1/servers/56dbfa6a-5b6f-4b82-977b-af735e6a6ac9" status: 200 len: 1809 microversion: 2.20 time: 0.173141 Aug 27 02:29:32.522591 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 346/695] 10.222.0.52 () {60 vars in 1277 bytes} [Sat Aug 27 02:29:32 2022] GET /compute/v2.1/servers/56dbfa6a-5b6f-4b82-977b-af735e6a6ac9 => generated 1809 bytes in 174 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 27 02:29:33.547163 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-13a121b0-63e0-4771-9245-647f90a0bb92 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Calling method '>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:29:33.553839 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-13a121b0-63e0-4771-9245-647f90a0bb92 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] [instance: 56dbfa6a-5b6f-4b82-977b-af735e6a6ac9] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:29:33.554551 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-13a121b0-63e0-4771-9245-647f90a0bb92 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:29:33.555107 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-13a121b0-63e0-4771-9245-647f90a0bb92 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:29:33.555563 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-13a121b0-63e0-4771-9245-647f90a0bb92 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:29:33.556292 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-13a121b0-63e0-4771-9245-647f90a0bb92 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:29:33.556749 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-13a121b0-63e0-4771-9245-647f90a0bb92 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:29:33.557222 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-13a121b0-63e0-4771-9245-647f90a0bb92 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:29:33.706968 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-13a121b0-63e0-4771-9245-647f90a0bb92 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:29:33.717376 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-13a121b0-63e0-4771-9245-647f90a0bb92 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:29:33.719508 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-13a121b0-63e0-4771-9245-647f90a0bb92 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:29:33.719508 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-13a121b0-63e0-4771-9245-647f90a0bb92 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] 10.222.0.52 "GET /compute/v2.1/servers/56dbfa6a-5b6f-4b82-977b-af735e6a6ac9" status: 200 len: 1809 microversion: 2.20 time: 0.177203 Aug 27 02:29:33.719938 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 350/696] 10.222.0.52 () {60 vars in 1277 bytes} [Sat Aug 27 02:29:33 2022] GET /compute/v2.1/servers/56dbfa6a-5b6f-4b82-977b-af735e6a6ac9 => generated 1809 bytes in 178 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 27 02:29:34.854410 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-8f000a2f-9e58-4103-aac1-e30a80292d8f tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:29:34.855388 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-8f000a2f-9e58-4103-aac1-e30a80292d8f tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] [instance: 56dbfa6a-5b6f-4b82-977b-af735e6a6ac9] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:29:34.861870 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-8f000a2f-9e58-4103-aac1-e30a80292d8f tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:29:34.862354 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-8f000a2f-9e58-4103-aac1-e30a80292d8f tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:29:34.862790 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-8f000a2f-9e58-4103-aac1-e30a80292d8f tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:29:34.863491 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-8f000a2f-9e58-4103-aac1-e30a80292d8f tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:29:34.863949 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-8f000a2f-9e58-4103-aac1-e30a80292d8f tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:29:34.864373 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-8f000a2f-9e58-4103-aac1-e30a80292d8f tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:29:35.010334 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-8f000a2f-9e58-4103-aac1-e30a80292d8f tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:29:35.019406 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-8f000a2f-9e58-4103-aac1-e30a80292d8f tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:29:35.020827 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-8f000a2f-9e58-4103-aac1-e30a80292d8f tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:29:35.022087 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-8f000a2f-9e58-4103-aac1-e30a80292d8f tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] 10.222.0.52 "GET /compute/v2.1/servers/56dbfa6a-5b6f-4b82-977b-af735e6a6ac9" status: 200 len: 1809 microversion: 2.20 time: 0.282517 Aug 27 02:29:35.023082 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 347/697] 10.222.0.52 () {60 vars in 1277 bytes} [Sat Aug 27 02:29:34 2022] GET /compute/v2.1/servers/56dbfa6a-5b6f-4b82-977b-af735e6a6ac9 => generated 1809 bytes in 284 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 27 02:29:36.046309 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-08cb7d54-92e4-453f-9c46-ffa2a10a9f3f tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Calling method '>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:29:36.047000 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-08cb7d54-92e4-453f-9c46-ffa2a10a9f3f tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] [instance: 56dbfa6a-5b6f-4b82-977b-af735e6a6ac9] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:29:36.054734 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-08cb7d54-92e4-453f-9c46-ffa2a10a9f3f tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:29:36.055264 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-08cb7d54-92e4-453f-9c46-ffa2a10a9f3f tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:29:36.055719 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-08cb7d54-92e4-453f-9c46-ffa2a10a9f3f tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:29:36.056434 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-08cb7d54-92e4-453f-9c46-ffa2a10a9f3f tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:29:36.056887 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-08cb7d54-92e4-453f-9c46-ffa2a10a9f3f tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:29:36.057345 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-08cb7d54-92e4-453f-9c46-ffa2a10a9f3f tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:29:36.199701 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-08cb7d54-92e4-453f-9c46-ffa2a10a9f3f tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:29:36.208391 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-08cb7d54-92e4-453f-9c46-ffa2a10a9f3f tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:29:36.210126 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-08cb7d54-92e4-453f-9c46-ffa2a10a9f3f tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:29:36.211694 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-08cb7d54-92e4-453f-9c46-ffa2a10a9f3f tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] 10.222.0.52 "GET /compute/v2.1/servers/56dbfa6a-5b6f-4b82-977b-af735e6a6ac9" status: 200 len: 1809 microversion: 2.20 time: 0.169244 Aug 27 02:29:36.212233 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 351/698] 10.222.0.52 () {60 vars in 1277 bytes} [Sat Aug 27 02:29:36 2022] GET /compute/v2.1/servers/56dbfa6a-5b6f-4b82-977b-af735e6a6ac9 => generated 1809 bytes in 170 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Aug 27 02:29:37.235217 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-4036ea0a-6c89-4f41-a11c-49017292ba9a tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:29:37.235824 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-4036ea0a-6c89-4f41-a11c-49017292ba9a tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] [instance: 56dbfa6a-5b6f-4b82-977b-af735e6a6ac9] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:29:37.238449 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Aug 27 02:29:37.238608 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: Sat Aug 27 02:29:37 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/56dbfa6a-5b6f-4b82-977b-af735e6a6ac9 (ip 10.222.0.52) !!! Aug 27 02:29:37.245170 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-4036ea0a-6c89-4f41-a11c-49017292ba9a tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:29:37.245649 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-4036ea0a-6c89-4f41-a11c-49017292ba9a tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:29:37.246076 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-4036ea0a-6c89-4f41-a11c-49017292ba9a tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:29:37.246770 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-4036ea0a-6c89-4f41-a11c-49017292ba9a tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:29:37.247218 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-4036ea0a-6c89-4f41-a11c-49017292ba9a tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:29:37.247655 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-4036ea0a-6c89-4f41-a11c-49017292ba9a tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:29:37.384974 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-4036ea0a-6c89-4f41-a11c-49017292ba9a tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:29:37.393685 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-4036ea0a-6c89-4f41-a11c-49017292ba9a tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:29:37.395110 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-4036ea0a-6c89-4f41-a11c-49017292ba9a tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:29:37.396251 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-4036ea0a-6c89-4f41-a11c-49017292ba9a tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] 10.222.0.52 "GET /compute/v2.1/servers/56dbfa6a-5b6f-4b82-977b-af735e6a6ac9" status: 200 len: 1809 microversion: 2.20 time: 0.165028 Aug 27 02:29:37.396774 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 348/699] 10.222.0.52 () {60 vars in 1277 bytes} [Sat Aug 27 02:29:37 2022] GET /compute/v2.1/servers/56dbfa6a-5b6f-4b82-977b-af735e6a6ac9 => generated 1809 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 27 02:29:37.759754 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-753f6f47-4165-42be-a5ba-eefef4faa09e tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Action: 'action', calling method: >, body: {"rescue": {"adminPass": "***"}} {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 27 02:29:37.762048 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-753f6f47-4165-42be-a5ba-eefef4faa09e tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] [instance: 98e4cfeb-d49e-4888-a2c0-40a9e8af90d3] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:29:37.769240 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-753f6f47-4165-42be-a5ba-eefef4faa09e tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:29:37.769712 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-753f6f47-4165-42be-a5ba-eefef4faa09e tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:29:37.770145 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-753f6f47-4165-42be-a5ba-eefef4faa09e tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:29:37.770825 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-753f6f47-4165-42be-a5ba-eefef4faa09e tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:29:37.771264 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-753f6f47-4165-42be-a5ba-eefef4faa09e tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:29:37.771675 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-753f6f47-4165-42be-a5ba-eefef4faa09e tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:29:37.796853 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.objects.instance [None req-753f6f47-4165-42be-a5ba-eefef4faa09e tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lazy-loading 'flavor' on Instance uuid 98e4cfeb-d49e-4888-a2c0-40a9e8af90d3 {{(pid=120864) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Aug 27 02:29:37.932167 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-753f6f47-4165-42be-a5ba-eefef4faa09e tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] 10.222.0.52 "POST /compute/v2.1/servers/98e4cfeb-d49e-4888-a2c0-40a9e8af90d3/action" status: 200 len: 32 microversion: 2.1 time: 0.176481 Aug 27 02:29:37.932781 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 352/700] 10.222.0.52 () {60 vars in 1278 bytes} [Sat Aug 27 02:29:37 2022] POST /compute/v2.1/servers/98e4cfeb-d49e-4888-a2c0-40a9e8af90d3/action => generated 32 bytes in 178 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Aug 27 02:29:37.942273 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-7fa4a9a5-3b4e-406b-bd27-8b5026a987af tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:29:37.943140 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-7fa4a9a5-3b4e-406b-bd27-8b5026a987af tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] [instance: 98e4cfeb-d49e-4888-a2c0-40a9e8af90d3] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:29:37.950070 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-7fa4a9a5-3b4e-406b-bd27-8b5026a987af tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:29:37.950840 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-7fa4a9a5-3b4e-406b-bd27-8b5026a987af tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:29:37.951248 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-7fa4a9a5-3b4e-406b-bd27-8b5026a987af tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:29:37.952130 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-7fa4a9a5-3b4e-406b-bd27-8b5026a987af tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:29:37.952746 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-7fa4a9a5-3b4e-406b-bd27-8b5026a987af tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:29:37.953392 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-7fa4a9a5-3b4e-406b-bd27-8b5026a987af tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:29:38.108356 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-7fa4a9a5-3b4e-406b-bd27-8b5026a987af tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd945d512ae034ec08838d0a3afde2429', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '880ac4003248441c809a76cfd1b221eb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:29:38.117349 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-7fa4a9a5-3b4e-406b-bd27-8b5026a987af tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] 10.222.0.52 "GET /compute/v2.1/servers/98e4cfeb-d49e-4888-a2c0-40a9e8af90d3" status: 200 len: 1735 microversion: 2.1 time: 0.178798 Aug 27 02:29:38.118040 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 349/701] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:29:37 2022] GET /compute/v2.1/servers/98e4cfeb-d49e-4888-a2c0-40a9e8af90d3 => generated 1735 bytes in 180 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 27 02:29:38.420822 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-7a381af2-0e78-445d-b7c1-67f5a79aa36d tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Calling method '>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:29:38.421456 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-7a381af2-0e78-445d-b7c1-67f5a79aa36d tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] [instance: 56dbfa6a-5b6f-4b82-977b-af735e6a6ac9] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:29:38.427777 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-7a381af2-0e78-445d-b7c1-67f5a79aa36d tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:29:38.428268 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-7a381af2-0e78-445d-b7c1-67f5a79aa36d tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:29:38.428703 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-7a381af2-0e78-445d-b7c1-67f5a79aa36d tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:29:38.429591 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-7a381af2-0e78-445d-b7c1-67f5a79aa36d tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:29:38.430046 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-7a381af2-0e78-445d-b7c1-67f5a79aa36d tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:29:38.430467 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-7a381af2-0e78-445d-b7c1-67f5a79aa36d tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:29:38.585817 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-7a381af2-0e78-445d-b7c1-67f5a79aa36d tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:29:38.595906 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-7a381af2-0e78-445d-b7c1-67f5a79aa36d tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:29:38.597481 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-7a381af2-0e78-445d-b7c1-67f5a79aa36d tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:29:38.599714 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-7a381af2-0e78-445d-b7c1-67f5a79aa36d tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] 10.222.0.52 "GET /compute/v2.1/servers/56dbfa6a-5b6f-4b82-977b-af735e6a6ac9" status: 200 len: 1809 microversion: 2.20 time: 0.182937 Aug 27 02:29:38.609303 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 353/702] 10.222.0.52 () {60 vars in 1277 bytes} [Sat Aug 27 02:29:38 2022] GET /compute/v2.1/servers/56dbfa6a-5b6f-4b82-977b-af735e6a6ac9 => generated 1809 bytes in 194 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 27 02:29:39.140017 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-cd29de54-ba4b-412a-a713-3ac12eff1c59 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:29:39.140712 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-cd29de54-ba4b-412a-a713-3ac12eff1c59 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] [instance: 98e4cfeb-d49e-4888-a2c0-40a9e8af90d3] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:29:39.148410 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-cd29de54-ba4b-412a-a713-3ac12eff1c59 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:29:39.149023 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-cd29de54-ba4b-412a-a713-3ac12eff1c59 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:29:39.149473 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-cd29de54-ba4b-412a-a713-3ac12eff1c59 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:29:39.150213 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-cd29de54-ba4b-412a-a713-3ac12eff1c59 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:29:39.150948 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-cd29de54-ba4b-412a-a713-3ac12eff1c59 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:29:39.151281 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-cd29de54-ba4b-412a-a713-3ac12eff1c59 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:29:39.291048 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-cd29de54-ba4b-412a-a713-3ac12eff1c59 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd945d512ae034ec08838d0a3afde2429', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '880ac4003248441c809a76cfd1b221eb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:29:39.299400 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-cd29de54-ba4b-412a-a713-3ac12eff1c59 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] 10.222.0.52 "GET /compute/v2.1/servers/98e4cfeb-d49e-4888-a2c0-40a9e8af90d3" status: 200 len: 1735 microversion: 2.1 time: 0.163969 Aug 27 02:29:39.299970 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 350/703] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:29:39 2022] GET /compute/v2.1/servers/98e4cfeb-d49e-4888-a2c0-40a9e8af90d3 => generated 1735 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:29:39.623447 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-3b73850b-813a-4aee-81d5-51ad7d3ba0a9 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Calling method '>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:29:39.624074 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-3b73850b-813a-4aee-81d5-51ad7d3ba0a9 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] [instance: 56dbfa6a-5b6f-4b82-977b-af735e6a6ac9] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:29:39.632477 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-3b73850b-813a-4aee-81d5-51ad7d3ba0a9 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:29:39.633379 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-3b73850b-813a-4aee-81d5-51ad7d3ba0a9 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:29:39.633831 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-3b73850b-813a-4aee-81d5-51ad7d3ba0a9 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:29:39.634539 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-3b73850b-813a-4aee-81d5-51ad7d3ba0a9 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:29:39.634993 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-3b73850b-813a-4aee-81d5-51ad7d3ba0a9 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:29:39.635414 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-3b73850b-813a-4aee-81d5-51ad7d3ba0a9 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:29:39.794689 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-3b73850b-813a-4aee-81d5-51ad7d3ba0a9 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:29:39.804091 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-3b73850b-813a-4aee-81d5-51ad7d3ba0a9 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:29:39.805669 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-3b73850b-813a-4aee-81d5-51ad7d3ba0a9 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:29:39.806849 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-3b73850b-813a-4aee-81d5-51ad7d3ba0a9 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] 10.222.0.52 "GET /compute/v2.1/servers/56dbfa6a-5b6f-4b82-977b-af735e6a6ac9" status: 200 len: 1809 microversion: 2.20 time: 0.187408 Aug 27 02:29:39.807404 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 354/704] 10.222.0.52 () {60 vars in 1277 bytes} [Sat Aug 27 02:29:39 2022] GET /compute/v2.1/servers/56dbfa6a-5b6f-4b82-977b-af735e6a6ac9 => generated 1809 bytes in 189 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 27 02:29:39.886482 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [req-857a6369-e5a7-4927-92de-5c467826cecb req-87b431a4-88c9-4eb6-ae1c-15e8f46a218e service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "56dbfa6a-5b6f-4b82-977b-af735e6a6ac9", "name": "network-vif-plugged", "status": "completed", "tag": "520bd5fb-723a-4570-83ca-72759c588707"}]} {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 27 02:29:39.898718 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [req-857a6369-e5a7-4927-92de-5c467826cecb req-87b431a4-88c9-4eb6-ae1c-15e8f46a218e service nova] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:29:39.899203 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [req-857a6369-e5a7-4927-92de-5c467826cecb req-87b431a4-88c9-4eb6-ae1c-15e8f46a218e service nova] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:29:39.899609 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [req-857a6369-e5a7-4927-92de-5c467826cecb req-87b431a4-88c9-4eb6-ae1c-15e8f46a218e service nova] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:29:39.911015 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.compute.server_external_events [req-857a6369-e5a7-4927-92de-5c467826cecb req-87b431a4-88c9-4eb6-ae1c-15e8f46a218e service nova] Creating event network-vif-plugged:520bd5fb-723a-4570-83ca-72759c588707 for instance 56dbfa6a-5b6f-4b82-977b-af735e6a6ac9 on hv-cinder-84630 Aug 27 02:29:39.916234 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [req-857a6369-e5a7-4927-92de-5c467826cecb req-87b431a4-88c9-4eb6-ae1c-15e8f46a218e service nova] 10.222.0.52 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.033554 Aug 27 02:29:39.916822 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 351/705] 10.222.0.52 () {64 vars in 1317 bytes} [Sat Aug 27 02:29:39 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 35 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Aug 27 02:29:40.322453 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-188b6693-24e6-456c-b9e3-59faecd6070e tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Calling method '>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:29:40.323101 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-188b6693-24e6-456c-b9e3-59faecd6070e tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] [instance: 98e4cfeb-d49e-4888-a2c0-40a9e8af90d3] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:29:40.329408 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-188b6693-24e6-456c-b9e3-59faecd6070e tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:29:40.329888 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-188b6693-24e6-456c-b9e3-59faecd6070e tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:29:40.330318 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-188b6693-24e6-456c-b9e3-59faecd6070e tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:29:40.331011 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-188b6693-24e6-456c-b9e3-59faecd6070e tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:29:40.331462 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-188b6693-24e6-456c-b9e3-59faecd6070e tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:29:40.331882 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-188b6693-24e6-456c-b9e3-59faecd6070e tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:29:40.437676 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-a43865fa-b381-4f93-a297-ef13b904d982 tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] Action: 'action', calling method: >, body: {"shelve": {}} {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 27 02:29:40.438608 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-a43865fa-b381-4f93-a297-ef13b904d982 tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] [instance: 9941c49f-8626-461d-a570-c0ffbfd4b804] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:29:40.445100 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-a43865fa-b381-4f93-a297-ef13b904d982 tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:29:40.445635 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-a43865fa-b381-4f93-a297-ef13b904d982 tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:29:40.446082 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-a43865fa-b381-4f93-a297-ef13b904d982 tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:29:40.446779 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-a43865fa-b381-4f93-a297-ef13b904d982 tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:29:40.447235 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-a43865fa-b381-4f93-a297-ef13b904d982 tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:29:40.452414 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-a43865fa-b381-4f93-a297-ef13b904d982 tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:29:40.479393 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.objects.instance [None req-a43865fa-b381-4f93-a297-ef13b904d982 tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] Lazy-loading 'flavor' on Instance uuid 9941c49f-8626-461d-a570-c0ffbfd4b804 {{(pid=120863) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Aug 27 02:29:40.479624 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-188b6693-24e6-456c-b9e3-59faecd6070e tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd945d512ae034ec08838d0a3afde2429', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '880ac4003248441c809a76cfd1b221eb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:29:40.489255 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-188b6693-24e6-456c-b9e3-59faecd6070e tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] 10.222.0.52 "GET /compute/v2.1/servers/98e4cfeb-d49e-4888-a2c0-40a9e8af90d3" status: 200 len: 1735 microversion: 2.1 time: 0.171753 Aug 27 02:29:40.489849 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 355/706] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:29:40 2022] GET /compute/v2.1/servers/98e4cfeb-d49e-4888-a2c0-40a9e8af90d3 => generated 1735 bytes in 173 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:29:40.595904 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-a43865fa-b381-4f93-a297-ef13b904d982 tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] 10.222.0.52 "POST /compute/v2.1/servers/9941c49f-8626-461d-a570-c0ffbfd4b804/action" status: 202 len: 0 microversion: 2.1 time: 0.162135 Aug 27 02:29:40.597064 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 352/707] 10.222.0.52 () {60 vars in 1278 bytes} [Sat Aug 27 02:29:40 2022] POST /compute/v2.1/servers/9941c49f-8626-461d-a570-c0ffbfd4b804/action => generated 0 bytes in 165 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Aug 27 02:29:40.606541 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-618381dc-62aa-4a26-915d-057ee3524e60 tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] Calling method '>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:29:40.607171 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-618381dc-62aa-4a26-915d-057ee3524e60 tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] [instance: 9941c49f-8626-461d-a570-c0ffbfd4b804] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:29:40.613972 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-618381dc-62aa-4a26-915d-057ee3524e60 tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:29:40.614339 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-618381dc-62aa-4a26-915d-057ee3524e60 tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:29:40.614837 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-618381dc-62aa-4a26-915d-057ee3524e60 tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:29:40.615694 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-618381dc-62aa-4a26-915d-057ee3524e60 tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:29:40.616205 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-618381dc-62aa-4a26-915d-057ee3524e60 tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:29:40.616826 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-618381dc-62aa-4a26-915d-057ee3524e60 tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:29:40.763802 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-618381dc-62aa-4a26-915d-057ee3524e60 tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '642291c4daa04a6ab3fbce92a8be39c6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b2ddff853b7546788b645a0344c2297b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:29:40.772571 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-618381dc-62aa-4a26-915d-057ee3524e60 tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] 10.222.0.52 "GET /compute/v2.1/servers/9941c49f-8626-461d-a570-c0ffbfd4b804" status: 200 len: 1569 microversion: 2.1 time: 0.169868 Aug 27 02:29:40.773404 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 356/708] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:29:40 2022] GET /compute/v2.1/servers/9941c49f-8626-461d-a570-c0ffbfd4b804 => generated 1569 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 27 02:29:40.835794 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-d3f574c2-63c1-42c7-aa02-380d51dea113 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:29:40.836740 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-d3f574c2-63c1-42c7-aa02-380d51dea113 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] [instance: 56dbfa6a-5b6f-4b82-977b-af735e6a6ac9] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:29:40.843476 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-d3f574c2-63c1-42c7-aa02-380d51dea113 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:29:40.843957 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-d3f574c2-63c1-42c7-aa02-380d51dea113 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:29:40.844395 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-d3f574c2-63c1-42c7-aa02-380d51dea113 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:29:40.845101 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-d3f574c2-63c1-42c7-aa02-380d51dea113 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:29:40.845590 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-d3f574c2-63c1-42c7-aa02-380d51dea113 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:29:40.846067 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-d3f574c2-63c1-42c7-aa02-380d51dea113 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:29:40.990088 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-d3f574c2-63c1-42c7-aa02-380d51dea113 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:29:40.999121 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-d3f574c2-63c1-42c7-aa02-380d51dea113 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:29:41.000543 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-d3f574c2-63c1-42c7-aa02-380d51dea113 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:29:41.001911 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-d3f574c2-63c1-42c7-aa02-380d51dea113 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] 10.222.0.52 "GET /compute/v2.1/servers/56dbfa6a-5b6f-4b82-977b-af735e6a6ac9" status: 200 len: 1809 microversion: 2.20 time: 0.170970 Aug 27 02:29:41.002462 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 353/709] 10.222.0.52 () {60 vars in 1277 bytes} [Sat Aug 27 02:29:40 2022] GET /compute/v2.1/servers/56dbfa6a-5b6f-4b82-977b-af735e6a6ac9 => generated 1809 bytes in 173 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 27 02:29:41.511813 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-dad6d3b1-1b4e-47e1-83bc-188c36774c25 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Calling method '>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:29:41.512453 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-dad6d3b1-1b4e-47e1-83bc-188c36774c25 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] [instance: 98e4cfeb-d49e-4888-a2c0-40a9e8af90d3] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:29:41.518945 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-dad6d3b1-1b4e-47e1-83bc-188c36774c25 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:29:41.519434 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-dad6d3b1-1b4e-47e1-83bc-188c36774c25 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:29:41.519866 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-dad6d3b1-1b4e-47e1-83bc-188c36774c25 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:29:41.520566 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-dad6d3b1-1b4e-47e1-83bc-188c36774c25 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:29:41.521024 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-dad6d3b1-1b4e-47e1-83bc-188c36774c25 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:29:41.521477 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-dad6d3b1-1b4e-47e1-83bc-188c36774c25 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:29:41.656240 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-dad6d3b1-1b4e-47e1-83bc-188c36774c25 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd945d512ae034ec08838d0a3afde2429', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '880ac4003248441c809a76cfd1b221eb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:29:41.665228 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-dad6d3b1-1b4e-47e1-83bc-188c36774c25 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] 10.222.0.52 "GET /compute/v2.1/servers/98e4cfeb-d49e-4888-a2c0-40a9e8af90d3" status: 200 len: 1735 microversion: 2.1 time: 0.157846 Aug 27 02:29:41.665798 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 357/710] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:29:41 2022] GET /compute/v2.1/servers/98e4cfeb-d49e-4888-a2c0-40a9e8af90d3 => generated 1735 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:29:41.798956 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-265c2854-4e6a-499f-8aa5-f3c286cee6ee tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:29:41.799944 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-265c2854-4e6a-499f-8aa5-f3c286cee6ee tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] [instance: 9941c49f-8626-461d-a570-c0ffbfd4b804] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:29:41.809773 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-265c2854-4e6a-499f-8aa5-f3c286cee6ee tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:29:41.810470 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-265c2854-4e6a-499f-8aa5-f3c286cee6ee tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:29:41.811100 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-265c2854-4e6a-499f-8aa5-f3c286cee6ee tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:29:41.812080 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-265c2854-4e6a-499f-8aa5-f3c286cee6ee tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:29:41.812694 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-265c2854-4e6a-499f-8aa5-f3c286cee6ee tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:29:41.813093 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-265c2854-4e6a-499f-8aa5-f3c286cee6ee tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:29:41.979246 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-265c2854-4e6a-499f-8aa5-f3c286cee6ee tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '642291c4daa04a6ab3fbce92a8be39c6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b2ddff853b7546788b645a0344c2297b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:29:41.987519 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-265c2854-4e6a-499f-8aa5-f3c286cee6ee tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] 10.222.0.52 "GET /compute/v2.1/servers/9941c49f-8626-461d-a570-c0ffbfd4b804" status: 200 len: 1569 microversion: 2.1 time: 0.195778 Aug 27 02:29:41.988091 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 354/711] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:29:41 2022] GET /compute/v2.1/servers/9941c49f-8626-461d-a570-c0ffbfd4b804 => generated 1569 bytes in 197 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:29:42.030732 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-119851c0-bd78-4e8d-84ba-be156e09d885 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Calling method '>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:29:42.031903 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-119851c0-bd78-4e8d-84ba-be156e09d885 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] [instance: 56dbfa6a-5b6f-4b82-977b-af735e6a6ac9] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:29:42.039888 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-119851c0-bd78-4e8d-84ba-be156e09d885 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:29:42.040355 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-119851c0-bd78-4e8d-84ba-be156e09d885 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:29:42.040855 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-119851c0-bd78-4e8d-84ba-be156e09d885 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:29:42.041533 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-119851c0-bd78-4e8d-84ba-be156e09d885 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:29:42.041976 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-119851c0-bd78-4e8d-84ba-be156e09d885 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:29:42.042425 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-119851c0-bd78-4e8d-84ba-be156e09d885 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:29:42.058434 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG dbcounter [-] [120864] Writing DB stats nova_cell1:SELECT=94,nova_cell1:UPDATE=4,nova_cell1:INSERT=2 {{(pid=120864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 27 02:29:42.188506 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-119851c0-bd78-4e8d-84ba-be156e09d885 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:29:42.198733 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-119851c0-bd78-4e8d-84ba-be156e09d885 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:29:42.200201 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-119851c0-bd78-4e8d-84ba-be156e09d885 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:29:42.201420 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-119851c0-bd78-4e8d-84ba-be156e09d885 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] 10.222.0.52 "GET /compute/v2.1/servers/56dbfa6a-5b6f-4b82-977b-af735e6a6ac9" status: 200 len: 1809 microversion: 2.20 time: 0.177903 Aug 27 02:29:42.201978 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 358/712] 10.222.0.52 () {60 vars in 1277 bytes} [Sat Aug 27 02:29:42 2022] GET /compute/v2.1/servers/56dbfa6a-5b6f-4b82-977b-af735e6a6ac9 => generated 1809 bytes in 180 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 27 02:29:42.687038 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-42666573-207b-4a45-9571-8081dcf30f6f tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:29:42.687652 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-42666573-207b-4a45-9571-8081dcf30f6f tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] [instance: 98e4cfeb-d49e-4888-a2c0-40a9e8af90d3] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:29:42.694287 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-42666573-207b-4a45-9571-8081dcf30f6f tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:29:42.694785 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-42666573-207b-4a45-9571-8081dcf30f6f tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:29:42.695389 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-42666573-207b-4a45-9571-8081dcf30f6f tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:29:42.696104 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-42666573-207b-4a45-9571-8081dcf30f6f tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:29:42.696556 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-42666573-207b-4a45-9571-8081dcf30f6f tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:29:42.696983 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-42666573-207b-4a45-9571-8081dcf30f6f tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:29:42.860128 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-42666573-207b-4a45-9571-8081dcf30f6f tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd945d512ae034ec08838d0a3afde2429', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '880ac4003248441c809a76cfd1b221eb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:29:42.870786 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-42666573-207b-4a45-9571-8081dcf30f6f tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] 10.222.0.52 "GET /compute/v2.1/servers/98e4cfeb-d49e-4888-a2c0-40a9e8af90d3" status: 200 len: 1735 microversion: 2.1 time: 0.187229 Aug 27 02:29:42.871373 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 355/713] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:29:42 2022] GET /compute/v2.1/servers/98e4cfeb-d49e-4888-a2c0-40a9e8af90d3 => generated 1735 bytes in 189 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:29:43.012824 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-2d3ac00b-80b0-4940-b93f-5b13185b5a0f tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] Calling method '>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:29:43.013475 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-2d3ac00b-80b0-4940-b93f-5b13185b5a0f tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] [instance: 9941c49f-8626-461d-a570-c0ffbfd4b804] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:29:43.020447 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-2d3ac00b-80b0-4940-b93f-5b13185b5a0f tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:29:43.020946 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-2d3ac00b-80b0-4940-b93f-5b13185b5a0f tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:29:43.021483 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-2d3ac00b-80b0-4940-b93f-5b13185b5a0f tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:29:43.022176 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-2d3ac00b-80b0-4940-b93f-5b13185b5a0f tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:29:43.022699 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-2d3ac00b-80b0-4940-b93f-5b13185b5a0f tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:29:43.023261 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-2d3ac00b-80b0-4940-b93f-5b13185b5a0f tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:29:43.167901 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-2d3ac00b-80b0-4940-b93f-5b13185b5a0f tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '642291c4daa04a6ab3fbce92a8be39c6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b2ddff853b7546788b645a0344c2297b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:29:43.175859 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-2d3ac00b-80b0-4940-b93f-5b13185b5a0f tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] 10.222.0.52 "GET /compute/v2.1/servers/9941c49f-8626-461d-a570-c0ffbfd4b804" status: 200 len: 1569 microversion: 2.1 time: 0.166986 Aug 27 02:29:43.176436 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 359/714] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:29:43 2022] GET /compute/v2.1/servers/9941c49f-8626-461d-a570-c0ffbfd4b804 => generated 1569 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 27 02:29:43.225591 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-47bec5ea-60ea-48fb-ad0c-243d08eeb99c tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:29:43.226254 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-47bec5ea-60ea-48fb-ad0c-243d08eeb99c tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] [instance: 56dbfa6a-5b6f-4b82-977b-af735e6a6ac9] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:29:43.233209 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-47bec5ea-60ea-48fb-ad0c-243d08eeb99c tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:29:43.234133 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-47bec5ea-60ea-48fb-ad0c-243d08eeb99c tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:29:43.234868 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-47bec5ea-60ea-48fb-ad0c-243d08eeb99c tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:29:43.235821 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-47bec5ea-60ea-48fb-ad0c-243d08eeb99c tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:29:43.236281 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-47bec5ea-60ea-48fb-ad0c-243d08eeb99c tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:29:43.236741 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-47bec5ea-60ea-48fb-ad0c-243d08eeb99c tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:29:43.253081 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG dbcounter [-] [120863] Writing DB stats nova_cell1:SELECT=97,nova_cell1:UPDATE=2,nova_cell1:INSERT=1 {{(pid=120863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 27 02:29:43.377537 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-47bec5ea-60ea-48fb-ad0c-243d08eeb99c tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:29:43.386284 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-47bec5ea-60ea-48fb-ad0c-243d08eeb99c tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:29:43.387710 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-47bec5ea-60ea-48fb-ad0c-243d08eeb99c tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:29:43.388868 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-47bec5ea-60ea-48fb-ad0c-243d08eeb99c tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] 10.222.0.52 "GET /compute/v2.1/servers/56dbfa6a-5b6f-4b82-977b-af735e6a6ac9" status: 200 len: 1796 microversion: 2.20 time: 0.167353 Aug 27 02:29:43.389431 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 356/715] 10.222.0.52 () {60 vars in 1277 bytes} [Sat Aug 27 02:29:43 2022] GET /compute/v2.1/servers/56dbfa6a-5b6f-4b82-977b-af735e6a6ac9 => generated 1796 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 27 02:29:43.893599 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-0a27819c-2aaf-4da1-a1c0-95fc2dd0a87a tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Calling method '>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:29:43.894195 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-0a27819c-2aaf-4da1-a1c0-95fc2dd0a87a tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] [instance: 98e4cfeb-d49e-4888-a2c0-40a9e8af90d3] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:29:43.901066 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-0a27819c-2aaf-4da1-a1c0-95fc2dd0a87a tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:29:43.901592 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-0a27819c-2aaf-4da1-a1c0-95fc2dd0a87a tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:29:43.902017 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-0a27819c-2aaf-4da1-a1c0-95fc2dd0a87a tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:29:43.902682 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-0a27819c-2aaf-4da1-a1c0-95fc2dd0a87a tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:29:43.903121 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-0a27819c-2aaf-4da1-a1c0-95fc2dd0a87a tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:29:43.903534 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-0a27819c-2aaf-4da1-a1c0-95fc2dd0a87a tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:29:44.037015 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-0a27819c-2aaf-4da1-a1c0-95fc2dd0a87a tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd945d512ae034ec08838d0a3afde2429', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '880ac4003248441c809a76cfd1b221eb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:29:44.045744 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-0a27819c-2aaf-4da1-a1c0-95fc2dd0a87a tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] 10.222.0.52 "GET /compute/v2.1/servers/98e4cfeb-d49e-4888-a2c0-40a9e8af90d3" status: 200 len: 1735 microversion: 2.1 time: 0.156269 Aug 27 02:29:44.046317 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 360/716] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:29:43 2022] GET /compute/v2.1/servers/98e4cfeb-d49e-4888-a2c0-40a9e8af90d3 => generated 1735 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:29:44.197649 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-12e26d51-85fc-4e57-a3b2-545c5282ff69 tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:29:44.198322 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-12e26d51-85fc-4e57-a3b2-545c5282ff69 tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] [instance: 9941c49f-8626-461d-a570-c0ffbfd4b804] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:29:44.200576 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG dbcounter [-] [120863] Writing DB stats nova_api:SELECT=40 {{(pid=120863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 27 02:29:44.206352 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-12e26d51-85fc-4e57-a3b2-545c5282ff69 tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:29:44.206848 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-12e26d51-85fc-4e57-a3b2-545c5282ff69 tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:29:44.207259 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-12e26d51-85fc-4e57-a3b2-545c5282ff69 tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:29:44.207950 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-12e26d51-85fc-4e57-a3b2-545c5282ff69 tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:29:44.208397 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-12e26d51-85fc-4e57-a3b2-545c5282ff69 tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:29:44.208812 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-12e26d51-85fc-4e57-a3b2-545c5282ff69 tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:29:44.346484 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-12e26d51-85fc-4e57-a3b2-545c5282ff69 tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '642291c4daa04a6ab3fbce92a8be39c6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b2ddff853b7546788b645a0344c2297b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:29:44.355488 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-12e26d51-85fc-4e57-a3b2-545c5282ff69 tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] 10.222.0.52 "GET /compute/v2.1/servers/9941c49f-8626-461d-a570-c0ffbfd4b804" status: 200 len: 1569 microversion: 2.1 time: 0.161980 Aug 27 02:29:44.356071 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 357/717] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:29:44 2022] GET /compute/v2.1/servers/9941c49f-8626-461d-a570-c0ffbfd4b804 => generated 1569 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:29:45.068716 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-a6ad827a-f014-4205-9618-4a8c9bf69577 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Calling method '>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:29:45.069323 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-a6ad827a-f014-4205-9618-4a8c9bf69577 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] [instance: 98e4cfeb-d49e-4888-a2c0-40a9e8af90d3] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:29:45.075976 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-a6ad827a-f014-4205-9618-4a8c9bf69577 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:29:45.076456 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-a6ad827a-f014-4205-9618-4a8c9bf69577 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:29:45.076882 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-a6ad827a-f014-4205-9618-4a8c9bf69577 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:29:45.077624 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-a6ad827a-f014-4205-9618-4a8c9bf69577 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:29:45.078063 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-a6ad827a-f014-4205-9618-4a8c9bf69577 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:29:45.078477 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-a6ad827a-f014-4205-9618-4a8c9bf69577 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:29:45.221771 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-a6ad827a-f014-4205-9618-4a8c9bf69577 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd945d512ae034ec08838d0a3afde2429', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '880ac4003248441c809a76cfd1b221eb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:29:45.231356 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-a6ad827a-f014-4205-9618-4a8c9bf69577 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] 10.222.0.52 "GET /compute/v2.1/servers/98e4cfeb-d49e-4888-a2c0-40a9e8af90d3" status: 200 len: 1735 microversion: 2.1 time: 0.166660 Aug 27 02:29:45.231905 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 361/718] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:29:45 2022] GET /compute/v2.1/servers/98e4cfeb-d49e-4888-a2c0-40a9e8af90d3 => generated 1735 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:29:45.377864 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-efdf2c2f-8040-458f-84f2-a89577932d7a tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:29:45.378463 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-efdf2c2f-8040-458f-84f2-a89577932d7a tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] [instance: 9941c49f-8626-461d-a570-c0ffbfd4b804] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:29:45.386670 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-efdf2c2f-8040-458f-84f2-a89577932d7a tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:29:45.387189 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-efdf2c2f-8040-458f-84f2-a89577932d7a tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:29:45.387606 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-efdf2c2f-8040-458f-84f2-a89577932d7a tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:29:45.388310 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-efdf2c2f-8040-458f-84f2-a89577932d7a tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:29:45.388794 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-efdf2c2f-8040-458f-84f2-a89577932d7a tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:29:45.389829 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-efdf2c2f-8040-458f-84f2-a89577932d7a tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:29:45.537293 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-efdf2c2f-8040-458f-84f2-a89577932d7a tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '642291c4daa04a6ab3fbce92a8be39c6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b2ddff853b7546788b645a0344c2297b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:29:45.553053 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-efdf2c2f-8040-458f-84f2-a89577932d7a tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] 10.222.0.52 "GET /compute/v2.1/servers/9941c49f-8626-461d-a570-c0ffbfd4b804" status: 200 len: 1569 microversion: 2.1 time: 0.178955 Aug 27 02:29:45.553662 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 358/719] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:29:45 2022] GET /compute/v2.1/servers/9941c49f-8626-461d-a570-c0ffbfd4b804 => generated 1569 bytes in 180 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:29:46.253562 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-b0b73b64-342d-4eca-92a5-a2ad04b07331 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Calling method '>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:29:46.254202 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-b0b73b64-342d-4eca-92a5-a2ad04b07331 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] [instance: 98e4cfeb-d49e-4888-a2c0-40a9e8af90d3] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:29:46.256455 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG dbcounter [-] [120864] Writing DB stats nova_api:SELECT=38 {{(pid=120864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 27 02:29:46.262454 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-b0b73b64-342d-4eca-92a5-a2ad04b07331 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:29:46.262928 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-b0b73b64-342d-4eca-92a5-a2ad04b07331 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:29:46.263373 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-b0b73b64-342d-4eca-92a5-a2ad04b07331 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:29:46.264189 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-b0b73b64-342d-4eca-92a5-a2ad04b07331 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:29:46.264636 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-b0b73b64-342d-4eca-92a5-a2ad04b07331 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:29:46.265059 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-b0b73b64-342d-4eca-92a5-a2ad04b07331 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:29:46.416409 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-b0b73b64-342d-4eca-92a5-a2ad04b07331 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd945d512ae034ec08838d0a3afde2429', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '880ac4003248441c809a76cfd1b221eb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:29:46.425647 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-b0b73b64-342d-4eca-92a5-a2ad04b07331 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] 10.222.0.52 "GET /compute/v2.1/servers/98e4cfeb-d49e-4888-a2c0-40a9e8af90d3" status: 200 len: 1735 microversion: 2.1 time: 0.175834 Aug 27 02:29:46.426197 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 362/720] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:29:46 2022] GET /compute/v2.1/servers/98e4cfeb-d49e-4888-a2c0-40a9e8af90d3 => generated 1735 bytes in 177 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:29:46.574891 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-f6518e4c-95fe-41b7-8d54-8a404ebb52a1 tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:29:46.577014 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-f6518e4c-95fe-41b7-8d54-8a404ebb52a1 tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] [instance: 9941c49f-8626-461d-a570-c0ffbfd4b804] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:29:46.582403 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-f6518e4c-95fe-41b7-8d54-8a404ebb52a1 tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:29:46.582877 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-f6518e4c-95fe-41b7-8d54-8a404ebb52a1 tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:29:46.583363 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-f6518e4c-95fe-41b7-8d54-8a404ebb52a1 tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:29:46.584065 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-f6518e4c-95fe-41b7-8d54-8a404ebb52a1 tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:29:46.584525 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-f6518e4c-95fe-41b7-8d54-8a404ebb52a1 tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:29:46.585297 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-f6518e4c-95fe-41b7-8d54-8a404ebb52a1 tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:29:46.738838 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-f6518e4c-95fe-41b7-8d54-8a404ebb52a1 tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '642291c4daa04a6ab3fbce92a8be39c6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b2ddff853b7546788b645a0344c2297b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:29:46.746934 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-f6518e4c-95fe-41b7-8d54-8a404ebb52a1 tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] 10.222.0.52 "GET /compute/v2.1/servers/9941c49f-8626-461d-a570-c0ffbfd4b804" status: 200 len: 1569 microversion: 2.1 time: 0.175919 Aug 27 02:29:46.747501 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 359/721] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:29:46 2022] GET /compute/v2.1/servers/9941c49f-8626-461d-a570-c0ffbfd4b804 => generated 1569 bytes in 177 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:29:47.447365 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-4dbef326-fa4c-4bfc-a90b-6f3d567afde9 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Calling method '>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:29:47.448192 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-4dbef326-fa4c-4bfc-a90b-6f3d567afde9 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] [instance: 98e4cfeb-d49e-4888-a2c0-40a9e8af90d3] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:29:47.454925 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-4dbef326-fa4c-4bfc-a90b-6f3d567afde9 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:29:47.455418 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-4dbef326-fa4c-4bfc-a90b-6f3d567afde9 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:29:47.455850 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-4dbef326-fa4c-4bfc-a90b-6f3d567afde9 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:29:47.456553 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-4dbef326-fa4c-4bfc-a90b-6f3d567afde9 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:29:47.457007 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-4dbef326-fa4c-4bfc-a90b-6f3d567afde9 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:29:47.457459 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-4dbef326-fa4c-4bfc-a90b-6f3d567afde9 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:29:47.601505 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-4dbef326-fa4c-4bfc-a90b-6f3d567afde9 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd945d512ae034ec08838d0a3afde2429', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '880ac4003248441c809a76cfd1b221eb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:29:47.610137 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-4dbef326-fa4c-4bfc-a90b-6f3d567afde9 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] 10.222.0.52 "GET /compute/v2.1/servers/98e4cfeb-d49e-4888-a2c0-40a9e8af90d3" status: 200 len: 1735 microversion: 2.1 time: 0.166770 Aug 27 02:29:47.610744 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 363/722] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:29:47 2022] GET /compute/v2.1/servers/98e4cfeb-d49e-4888-a2c0-40a9e8af90d3 => generated 1735 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:29:47.769057 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-84253e34-77ef-46f8-a587-a1654a4fd0c1 tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:29:47.769706 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-84253e34-77ef-46f8-a587-a1654a4fd0c1 tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] [instance: 9941c49f-8626-461d-a570-c0ffbfd4b804] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:29:47.777926 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-84253e34-77ef-46f8-a587-a1654a4fd0c1 tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:29:47.778528 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-84253e34-77ef-46f8-a587-a1654a4fd0c1 tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:29:47.779021 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-84253e34-77ef-46f8-a587-a1654a4fd0c1 tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:29:47.779925 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-84253e34-77ef-46f8-a587-a1654a4fd0c1 tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:29:47.781016 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-84253e34-77ef-46f8-a587-a1654a4fd0c1 tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:29:47.781585 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-84253e34-77ef-46f8-a587-a1654a4fd0c1 tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:29:47.933016 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-84253e34-77ef-46f8-a587-a1654a4fd0c1 tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '642291c4daa04a6ab3fbce92a8be39c6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b2ddff853b7546788b645a0344c2297b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:29:47.942443 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-84253e34-77ef-46f8-a587-a1654a4fd0c1 tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] 10.222.0.52 "GET /compute/v2.1/servers/9941c49f-8626-461d-a570-c0ffbfd4b804" status: 200 len: 1569 microversion: 2.1 time: 0.178163 Aug 27 02:29:47.942947 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 360/723] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:29:47 2022] GET /compute/v2.1/servers/9941c49f-8626-461d-a570-c0ffbfd4b804 => generated 1569 bytes in 179 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:29:48.634862 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-0111c6d4-3bb1-43f2-92d7-e12aee0c1212 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Calling method '>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:29:48.635433 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-0111c6d4-3bb1-43f2-92d7-e12aee0c1212 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] [instance: 98e4cfeb-d49e-4888-a2c0-40a9e8af90d3] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:29:48.642132 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-0111c6d4-3bb1-43f2-92d7-e12aee0c1212 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:29:48.642606 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-0111c6d4-3bb1-43f2-92d7-e12aee0c1212 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:29:48.643030 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-0111c6d4-3bb1-43f2-92d7-e12aee0c1212 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:29:48.643709 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-0111c6d4-3bb1-43f2-92d7-e12aee0c1212 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:29:48.644155 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-0111c6d4-3bb1-43f2-92d7-e12aee0c1212 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:29:48.644566 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-0111c6d4-3bb1-43f2-92d7-e12aee0c1212 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:29:48.784195 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-0111c6d4-3bb1-43f2-92d7-e12aee0c1212 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd945d512ae034ec08838d0a3afde2429', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '880ac4003248441c809a76cfd1b221eb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:29:48.793231 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-0111c6d4-3bb1-43f2-92d7-e12aee0c1212 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] 10.222.0.52 "GET /compute/v2.1/servers/98e4cfeb-d49e-4888-a2c0-40a9e8af90d3" status: 200 len: 1735 microversion: 2.1 time: 0.163607 Aug 27 02:29:48.793801 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 364/724] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:29:48 2022] GET /compute/v2.1/servers/98e4cfeb-d49e-4888-a2c0-40a9e8af90d3 => generated 1735 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:29:48.965918 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-545eede5-cd59-49ee-9031-c03d818a6416 tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:29:48.966480 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-545eede5-cd59-49ee-9031-c03d818a6416 tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] [instance: 9941c49f-8626-461d-a570-c0ffbfd4b804] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:29:48.973314 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-545eede5-cd59-49ee-9031-c03d818a6416 tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:29:48.973794 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-545eede5-cd59-49ee-9031-c03d818a6416 tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:29:48.974220 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-545eede5-cd59-49ee-9031-c03d818a6416 tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:29:48.974908 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-545eede5-cd59-49ee-9031-c03d818a6416 tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:29:48.975340 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-545eede5-cd59-49ee-9031-c03d818a6416 tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:29:48.975755 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-545eede5-cd59-49ee-9031-c03d818a6416 tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:29:49.118765 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-545eede5-cd59-49ee-9031-c03d818a6416 tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '642291c4daa04a6ab3fbce92a8be39c6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b2ddff853b7546788b645a0344c2297b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:29:49.127231 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-545eede5-cd59-49ee-9031-c03d818a6416 tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] 10.222.0.52 "GET /compute/v2.1/servers/9941c49f-8626-461d-a570-c0ffbfd4b804" status: 200 len: 1569 microversion: 2.1 time: 0.165323 Aug 27 02:29:49.127797 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 361/725] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:29:48 2022] GET /compute/v2.1/servers/9941c49f-8626-461d-a570-c0ffbfd4b804 => generated 1569 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:29:49.815014 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-aeb14f15-312c-4d63-9d2d-c0d21bca9a6f tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Calling method '>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:29:49.815641 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-aeb14f15-312c-4d63-9d2d-c0d21bca9a6f tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] [instance: 98e4cfeb-d49e-4888-a2c0-40a9e8af90d3] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:29:49.822168 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-aeb14f15-312c-4d63-9d2d-c0d21bca9a6f tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:29:49.822646 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-aeb14f15-312c-4d63-9d2d-c0d21bca9a6f tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:29:49.823073 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-aeb14f15-312c-4d63-9d2d-c0d21bca9a6f tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:29:49.823761 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-aeb14f15-312c-4d63-9d2d-c0d21bca9a6f tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:29:49.824210 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-aeb14f15-312c-4d63-9d2d-c0d21bca9a6f tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:29:49.824613 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-aeb14f15-312c-4d63-9d2d-c0d21bca9a6f tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:29:49.966387 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-aeb14f15-312c-4d63-9d2d-c0d21bca9a6f tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd945d512ae034ec08838d0a3afde2429', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '880ac4003248441c809a76cfd1b221eb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:29:49.975213 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-aeb14f15-312c-4d63-9d2d-c0d21bca9a6f tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] 10.222.0.52 "GET /compute/v2.1/servers/98e4cfeb-d49e-4888-a2c0-40a9e8af90d3" status: 200 len: 1735 microversion: 2.1 time: 0.163896 Aug 27 02:29:49.975601 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 365/726] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:29:49 2022] GET /compute/v2.1/servers/98e4cfeb-d49e-4888-a2c0-40a9e8af90d3 => generated 1735 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:29:50.152021 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-08d87aea-97fd-4f9a-8fee-f18869ff07b3 tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:29:50.152641 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-08d87aea-97fd-4f9a-8fee-f18869ff07b3 tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] [instance: 9941c49f-8626-461d-a570-c0ffbfd4b804] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:29:50.160367 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-08d87aea-97fd-4f9a-8fee-f18869ff07b3 tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:29:50.160852 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-08d87aea-97fd-4f9a-8fee-f18869ff07b3 tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:29:50.161355 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-08d87aea-97fd-4f9a-8fee-f18869ff07b3 tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:29:50.162046 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-08d87aea-97fd-4f9a-8fee-f18869ff07b3 tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:29:50.162492 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-08d87aea-97fd-4f9a-8fee-f18869ff07b3 tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:29:50.162908 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-08d87aea-97fd-4f9a-8fee-f18869ff07b3 tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:29:50.302473 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-08d87aea-97fd-4f9a-8fee-f18869ff07b3 tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '642291c4daa04a6ab3fbce92a8be39c6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b2ddff853b7546788b645a0344c2297b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:29:50.311958 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-08d87aea-97fd-4f9a-8fee-f18869ff07b3 tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] 10.222.0.52 "GET /compute/v2.1/servers/9941c49f-8626-461d-a570-c0ffbfd4b804" status: 200 len: 1569 microversion: 2.1 time: 0.163771 Aug 27 02:29:50.312558 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 362/727] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:29:50 2022] GET /compute/v2.1/servers/9941c49f-8626-461d-a570-c0ffbfd4b804 => generated 1569 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:29:50.343869 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-85ab88fe-c30a-4f41-89d6-85defab27654 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Calling method '>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:29:50.344348 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-85ab88fe-c30a-4f41-89d6-85defab27654 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] [instance: dc699df0-be4f-4036-9e01-45ba0156616b] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:29:50.350647 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-85ab88fe-c30a-4f41-89d6-85defab27654 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:29:50.351107 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-85ab88fe-c30a-4f41-89d6-85defab27654 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:29:50.351703 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-85ab88fe-c30a-4f41-89d6-85defab27654 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:29:50.352397 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-85ab88fe-c30a-4f41-89d6-85defab27654 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:29:50.352855 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-85ab88fe-c30a-4f41-89d6-85defab27654 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:29:50.353318 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-85ab88fe-c30a-4f41-89d6-85defab27654 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:29:50.376572 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-85ab88fe-c30a-4f41-89d6-85defab27654 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] 10.222.0.52 "GET /compute/v2.1/servers/dc699df0-be4f-4036-9e01-45ba0156616b/ips" status: 200 len: 149 microversion: 2.1 time: 0.147000 Aug 27 02:29:50.377214 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 366/728] 10.222.0.52 () {58 vars in 1248 bytes} [Sat Aug 27 02:29:50 2022] GET /compute/v2.1/servers/dc699df0-be4f-4036-9e01-45ba0156616b/ips => generated 149 bytes in 148 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 27 02:29:50.389825 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-aaed4dc3-e5b1-4f1e-aa27-6ed5d1934249 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "58ee6df8-0ef2-453b-82ec-e4643432a653"}} {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 27 02:29:50.392250 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-aaed4dc3-e5b1-4f1e-aa27-6ed5d1934249 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] [instance: dc699df0-be4f-4036-9e01-45ba0156616b] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:29:50.399631 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-aaed4dc3-e5b1-4f1e-aa27-6ed5d1934249 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:29:50.400118 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-aaed4dc3-e5b1-4f1e-aa27-6ed5d1934249 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:29:50.400554 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-aaed4dc3-e5b1-4f1e-aa27-6ed5d1934249 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:29:50.401295 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-aaed4dc3-e5b1-4f1e-aa27-6ed5d1934249 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:29:50.401746 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-aaed4dc3-e5b1-4f1e-aa27-6ed5d1934249 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:29:50.402157 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-aaed4dc3-e5b1-4f1e-aa27-6ed5d1934249 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:29:50.488420 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.wsgi [None req-aaed4dc3-e5b1-4f1e-aa27-6ed5d1934249 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] HTTP exception thrown: Invalid volume: volume 58ee6df8-0ef2-453b-82ec-e4643432a653 is already attached to instances: 2ad09ab6-ef1f-47a2-bf44-6ae046cc8645 Aug 27 02:29:50.490121 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-aaed4dc3-e5b1-4f1e-aa27-6ed5d1934249 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Request method failure captured: Aug 27 02:29:50.490121 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: request: POST /compute/v2.1/servers/dc699df0-be4f-4036-9e01-45ba0156616b/os-volume_attachments HTTP/1.1 Aug 27 02:29:50.490121 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: Accept: application/json Aug 27 02:29:50.490121 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: Accept-Encoding: identity Aug 27 02:29:50.490121 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: Connection: close Aug 27 02:29:50.490121 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: Content-Length: 74 Aug 27 02:29:50.490121 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: Content-Type: application/json Aug 27 02:29:50.490121 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: Host: 10.222.0.52 Aug 27 02:29:50.490121 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: Openstack-System-Scope: None Aug 27 02:29:50.490121 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: User-Agent: python-urllib3/1.26.11 Aug 27 02:29:50.490121 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-Auth-Token: gAAAAABjCX_pkOb4-Fffq6Hbe0cfjlj1ARL1JmORSzrYWucWmWgZ67LEorDD3IWpPkj3UP534US8GW_nja0iAKwcdxSx3e5teQzZMdrCvOvA5aSGJO8iodOjhfOIsl8L0d8om8kj8_3q-8uTHxKupSTnIQy_J-f1kC4e22VNejIIiExDsGitats Aug 27 02:29:50.490121 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-Domain-Id: None Aug 27 02:29:50.490121 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-Domain-Name: None Aug 27 02:29:50.490121 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-Identity-Status: Confirmed Aug 27 02:29:50.490121 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-Is-Admin-Project: True Aug 27 02:29:50.490121 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-Project-Domain-Id: default Aug 27 02:29:50.490121 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-Project-Domain-Name: Default Aug 27 02:29:50.490121 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-Project-Id: 5d5ec8450c0941d18d63558baab54737 Aug 27 02:29:50.490121 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-Project-Name: tempest-AttachVolumeNegativeTest-1988669364 Aug 27 02:29:50.490121 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-Role: reader,member Aug 27 02:29:50.490121 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-Roles: reader,member Aug 27 02:29:50.492069 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-Service-Catalog: [{"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.52:8080", "publicURL": "http://10.222.0.52:8080/v1/AUTH_5d5ec8450c0941d18d63558baab54737"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52:9696/networking"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/compute/v2/5d5ec8450c0941d18d63558baab54737"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/image"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/volume/v3/5d5ec8450c0941d18d63558baab54737"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/identity"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/volume/v3/5d5ec8450c0941d18d63558baab54737"}]}] Aug 27 02:29:50.492069 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-Tenant: tempest-AttachVolumeNegativeTest-1988669364 Aug 27 02:29:50.492069 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-Tenant-Id: 5d5ec8450c0941d18d63558baab54737 Aug 27 02:29:50.492069 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-1988669364 Aug 27 02:29:50.492069 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-User: tempest-AttachVolumeNegativeTest-1988669364-project Aug 27 02:29:50.492069 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-User-Domain-Id: default Aug 27 02:29:50.492069 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-User-Domain-Name: Default Aug 27 02:29:50.492069 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-User-Id: 26a86351583d49d090b3db09f127d49f Aug 27 02:29:50.492069 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-User-Name: tempest-AttachVolumeNegativeTest-1988669364-project Aug 27 02:29:50.492069 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: Aug 27 02:29:50.492069 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: {"volumeAttachment": {"volumeId": "58ee6df8-0ef2-453b-82ec-e4643432a653"}} Aug 27 02:29:50.492069 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: method: > Aug 27 02:29:50.493856 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: exception: Invalid volume: volume 58ee6df8-0ef2-453b-82ec-e4643432a653 is already attached to instances: 2ad09ab6-ef1f-47a2-bf44-6ae046cc8645 Aug 27 02:29:50.493856 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: action_args: {'server_id': 'dc699df0-be4f-4036-9e01-45ba0156616b', 'body': {'volumeAttachment': {'volumeId': '58ee6df8-0ef2-453b-82ec-e4643432a653'}}} Aug 27 02:29:50.493856 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Aug 27 02:29:50.493856 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 27 02:29:50.493856 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 388, in create Aug 27 02:29:50.493856 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi device = self.compute_api.attach_volume( Aug 27 02:29:50.493856 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 241, in inner Aug 27 02:29:50.493856 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Aug 27 02:29:50.493856 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 167, in inner Aug 27 02:29:50.493856 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi return f(self, context, instance, *args, **kw) Aug 27 02:29:50.493856 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5128, in attach_volume Aug 27 02:29:50.493856 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi self._check_volume_already_attached(context, instance, volume) Aug 27 02:29:50.493856 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 4968, in _check_volume_already_attached Aug 27 02:29:50.493856 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi raise exception.InvalidVolume(reason=msg) Aug 27 02:29:50.493856 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi nova.exception.InvalidVolume: Invalid volume: volume 58ee6df8-0ef2-453b-82ec-e4643432a653 is already attached to instances: 2ad09ab6-ef1f-47a2-bf44-6ae046cc8645 Aug 27 02:29:50.493856 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi Aug 27 02:29:50.493856 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 27 02:29:50.493856 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi Aug 27 02:29:50.493856 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 27 02:29:50.495250 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 27 02:29:50.495250 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 27 02:29:50.495250 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Aug 27 02:29:50.495250 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Aug 27 02:29:50.495250 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Aug 27 02:29:50.495250 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Aug 27 02:29:50.495250 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Aug 27 02:29:50.495250 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Aug 27 02:29:50.495250 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Aug 27 02:29:50.495250 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Aug 27 02:29:50.495250 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Aug 27 02:29:50.495250 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Aug 27 02:29:50.495250 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 406, in create Aug 27 02:29:50.495250 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi raise exc.HTTPBadRequest(explanation=e.format_message()) Aug 27 02:29:50.495250 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi webob.exc.HTTPBadRequest: Invalid volume: volume 58ee6df8-0ef2-453b-82ec-e4643432a653 is already attached to instances: 2ad09ab6-ef1f-47a2-bf44-6ae046cc8645 Aug 27 02:29:50.495250 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi Aug 27 02:29:50.495250 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 27 02:29:50.495250 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi Aug 27 02:29:50.495250 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 27 02:29:50.495250 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 27 02:29:50.496908 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 27 02:29:50.496908 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Aug 27 02:29:50.496908 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Aug 27 02:29:50.496908 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid volume: volume 58ee6df8-0ef2-453b-82ec-e4643432a653 is already attached to instances: 2ad09ab6-ef1f-47a2-bf44-6ae046cc8645 Aug 27 02:29:50.496908 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi Aug 27 02:29:50.496908 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-aaed4dc3-e5b1-4f1e-aa27-6ed5d1934249 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Returning 400 to user: Invalid volume: volume 58ee6df8-0ef2-453b-82ec-e4643432a653 is already attached to instances: 2ad09ab6-ef1f-47a2-bf44-6ae046cc8645 {{(pid=120863) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Aug 27 02:29:50.496908 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-aaed4dc3-e5b1-4f1e-aa27-6ed5d1934249 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] 10.222.0.52 "POST /compute/v2.1/servers/dc699df0-be4f-4036-9e01-45ba0156616b/os-volume_attachments" status: 400 len: 174 microversion: 2.1 time: 0.106033 Aug 27 02:29:50.496908 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 363/729] 10.222.0.52 () {60 vars in 1323 bytes} [Sat Aug 27 02:29:50 2022] POST /compute/v2.1/servers/dc699df0-be4f-4036-9e01-45ba0156616b/os-volume_attachments => generated 174 bytes in 107 msecs (HTTP/1.1 400) 9 headers in 381 bytes (1 switches on core 0) Aug 27 02:29:50.500884 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-a04adb5f-e097-4d90-89b8-31dbfca2909b tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Calling method '>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:29:50.501447 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-a04adb5f-e097-4d90-89b8-31dbfca2909b tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] [instance: dc699df0-be4f-4036-9e01-45ba0156616b] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:29:50.507762 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-a04adb5f-e097-4d90-89b8-31dbfca2909b tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:29:50.508251 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-a04adb5f-e097-4d90-89b8-31dbfca2909b tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:29:50.508905 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-a04adb5f-e097-4d90-89b8-31dbfca2909b tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:29:50.509636 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-a04adb5f-e097-4d90-89b8-31dbfca2909b tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:29:50.510090 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-a04adb5f-e097-4d90-89b8-31dbfca2909b tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:29:50.510513 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-a04adb5f-e097-4d90-89b8-31dbfca2909b tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:29:50.537971 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-a04adb5f-e097-4d90-89b8-31dbfca2909b tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] [instance: dc699df0-be4f-4036-9e01-45ba0156616b] Going to try to terminate instance {{(pid=120864) delete /opt/stack/nova/nova/compute/api.py:2719}} Aug 27 02:29:50.623820 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-a04adb5f-e097-4d90-89b8-31dbfca2909b tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] 10.222.0.52 "DELETE /compute/v2.1/servers/dc699df0-be4f-4036-9e01-45ba0156616b" status: 204 len: 0 microversion: 2.1 time: 0.126580 Aug 27 02:29:50.624378 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 367/730] 10.222.0.52 () {58 vars in 1239 bytes} [Sat Aug 27 02:29:50 2022] DELETE /compute/v2.1/servers/dc699df0-be4f-4036-9e01-45ba0156616b => generated 0 bytes in 128 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Aug 27 02:29:50.632925 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-dd7112f9-dfbc-4ba3-9fbf-27cbd6e7513b tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:29:50.633749 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-dd7112f9-dfbc-4ba3-9fbf-27cbd6e7513b tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] [instance: dc699df0-be4f-4036-9e01-45ba0156616b] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:29:50.641339 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-dd7112f9-dfbc-4ba3-9fbf-27cbd6e7513b tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:29:50.641845 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-dd7112f9-dfbc-4ba3-9fbf-27cbd6e7513b tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:29:50.642330 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-dd7112f9-dfbc-4ba3-9fbf-27cbd6e7513b tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:29:50.643042 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-dd7112f9-dfbc-4ba3-9fbf-27cbd6e7513b tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:29:50.643662 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-dd7112f9-dfbc-4ba3-9fbf-27cbd6e7513b tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:29:50.643926 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-dd7112f9-dfbc-4ba3-9fbf-27cbd6e7513b tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:29:50.789968 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-dd7112f9-dfbc-4ba3-9fbf-27cbd6e7513b tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '26a86351583d49d090b3db09f127d49f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5d5ec8450c0941d18d63558baab54737', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:29:50.797756 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-dd7112f9-dfbc-4ba3-9fbf-27cbd6e7513b tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] 10.222.0.52 "GET /compute/v2.1/servers/dc699df0-be4f-4036-9e01-45ba0156616b" status: 200 len: 1680 microversion: 2.1 time: 0.169039 Aug 27 02:29:50.798343 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 364/731] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:29:50 2022] GET /compute/v2.1/servers/dc699df0-be4f-4036-9e01-45ba0156616b => generated 1680 bytes in 170 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:29:50.998035 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-e814fd2d-8acd-4f4a-a594-bb8bf72ed0d0 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Calling method '>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:29:50.998703 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-e814fd2d-8acd-4f4a-a594-bb8bf72ed0d0 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] [instance: 98e4cfeb-d49e-4888-a2c0-40a9e8af90d3] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:29:51.005451 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-e814fd2d-8acd-4f4a-a594-bb8bf72ed0d0 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:29:51.005939 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-e814fd2d-8acd-4f4a-a594-bb8bf72ed0d0 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:29:51.006373 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-e814fd2d-8acd-4f4a-a594-bb8bf72ed0d0 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:29:51.007072 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-e814fd2d-8acd-4f4a-a594-bb8bf72ed0d0 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:29:51.007656 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-e814fd2d-8acd-4f4a-a594-bb8bf72ed0d0 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:29:51.008397 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-e814fd2d-8acd-4f4a-a594-bb8bf72ed0d0 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:29:51.143962 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-e814fd2d-8acd-4f4a-a594-bb8bf72ed0d0 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd945d512ae034ec08838d0a3afde2429', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '880ac4003248441c809a76cfd1b221eb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:29:51.154267 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-e814fd2d-8acd-4f4a-a594-bb8bf72ed0d0 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] 10.222.0.52 "GET /compute/v2.1/servers/98e4cfeb-d49e-4888-a2c0-40a9e8af90d3" status: 200 len: 1735 microversion: 2.1 time: 0.160474 Aug 27 02:29:51.154868 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 368/732] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:29:50 2022] GET /compute/v2.1/servers/98e4cfeb-d49e-4888-a2c0-40a9e8af90d3 => generated 1735 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:29:51.337773 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-ea7709ba-77ca-432d-8c51-4b07bff5d6b0 tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:29:51.338452 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-ea7709ba-77ca-432d-8c51-4b07bff5d6b0 tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] [instance: 9941c49f-8626-461d-a570-c0ffbfd4b804] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:29:51.344944 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-ea7709ba-77ca-432d-8c51-4b07bff5d6b0 tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:29:51.345493 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-ea7709ba-77ca-432d-8c51-4b07bff5d6b0 tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:29:51.345926 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-ea7709ba-77ca-432d-8c51-4b07bff5d6b0 tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:29:51.346612 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-ea7709ba-77ca-432d-8c51-4b07bff5d6b0 tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:29:51.347054 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-ea7709ba-77ca-432d-8c51-4b07bff5d6b0 tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:29:51.347467 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-ea7709ba-77ca-432d-8c51-4b07bff5d6b0 tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:29:51.491881 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-ea7709ba-77ca-432d-8c51-4b07bff5d6b0 tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '642291c4daa04a6ab3fbce92a8be39c6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b2ddff853b7546788b645a0344c2297b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:29:51.500114 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-ea7709ba-77ca-432d-8c51-4b07bff5d6b0 tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] 10.222.0.52 "GET /compute/v2.1/servers/9941c49f-8626-461d-a570-c0ffbfd4b804" status: 200 len: 1569 microversion: 2.1 time: 0.168942 Aug 27 02:29:51.500653 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 365/733] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:29:51 2022] GET /compute/v2.1/servers/9941c49f-8626-461d-a570-c0ffbfd4b804 => generated 1569 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:29:51.821792 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-406571a1-1736-4db5-9f18-ee054c6629b3 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Calling method '>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:29:51.822398 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-406571a1-1736-4db5-9f18-ee054c6629b3 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] [instance: dc699df0-be4f-4036-9e01-45ba0156616b] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:29:51.828563 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-406571a1-1736-4db5-9f18-ee054c6629b3 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:29:51.829046 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-406571a1-1736-4db5-9f18-ee054c6629b3 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:29:51.829509 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-406571a1-1736-4db5-9f18-ee054c6629b3 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:29:51.830190 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-406571a1-1736-4db5-9f18-ee054c6629b3 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:29:51.830641 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-406571a1-1736-4db5-9f18-ee054c6629b3 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:29:51.831061 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-406571a1-1736-4db5-9f18-ee054c6629b3 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:29:51.966786 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-406571a1-1736-4db5-9f18-ee054c6629b3 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '26a86351583d49d090b3db09f127d49f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5d5ec8450c0941d18d63558baab54737', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:29:51.984946 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-406571a1-1736-4db5-9f18-ee054c6629b3 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] 10.222.0.52 "GET /compute/v2.1/servers/dc699df0-be4f-4036-9e01-45ba0156616b" status: 200 len: 1680 microversion: 2.1 time: 0.169162 Aug 27 02:29:51.985483 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 369/734] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:29:51 2022] GET /compute/v2.1/servers/dc699df0-be4f-4036-9e01-45ba0156616b => generated 1680 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:29:52.177893 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-f6ccff89-1509-4b45-9556-09820d261a9b tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:29:52.178859 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-f6ccff89-1509-4b45-9556-09820d261a9b tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] [instance: 98e4cfeb-d49e-4888-a2c0-40a9e8af90d3] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:29:52.185386 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-f6ccff89-1509-4b45-9556-09820d261a9b tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:29:52.185896 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-f6ccff89-1509-4b45-9556-09820d261a9b tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:29:52.186314 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-f6ccff89-1509-4b45-9556-09820d261a9b tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:29:52.187011 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-f6ccff89-1509-4b45-9556-09820d261a9b tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:29:52.187463 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-f6ccff89-1509-4b45-9556-09820d261a9b tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:29:52.187885 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-f6ccff89-1509-4b45-9556-09820d261a9b tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:29:52.326886 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-f6ccff89-1509-4b45-9556-09820d261a9b tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd945d512ae034ec08838d0a3afde2429', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '880ac4003248441c809a76cfd1b221eb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:29:52.335713 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-f6ccff89-1509-4b45-9556-09820d261a9b tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] 10.222.0.52 "GET /compute/v2.1/servers/98e4cfeb-d49e-4888-a2c0-40a9e8af90d3" status: 200 len: 1735 microversion: 2.1 time: 0.163032 Aug 27 02:29:52.336468 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 366/735] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:29:52 2022] GET /compute/v2.1/servers/98e4cfeb-d49e-4888-a2c0-40a9e8af90d3 => generated 1735 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:29:52.521992 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-e6faddcf-ecf2-45ad-a7d6-d4634b9603d0 tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] Calling method '>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:29:52.522603 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-e6faddcf-ecf2-45ad-a7d6-d4634b9603d0 tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] [instance: 9941c49f-8626-461d-a570-c0ffbfd4b804] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:29:52.529624 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-e6faddcf-ecf2-45ad-a7d6-d4634b9603d0 tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:29:52.530132 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-e6faddcf-ecf2-45ad-a7d6-d4634b9603d0 tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:29:52.530582 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-e6faddcf-ecf2-45ad-a7d6-d4634b9603d0 tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:29:52.531426 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-e6faddcf-ecf2-45ad-a7d6-d4634b9603d0 tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:29:52.531947 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-e6faddcf-ecf2-45ad-a7d6-d4634b9603d0 tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:29:52.532396 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-e6faddcf-ecf2-45ad-a7d6-d4634b9603d0 tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:29:52.789530 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-e6faddcf-ecf2-45ad-a7d6-d4634b9603d0 tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '642291c4daa04a6ab3fbce92a8be39c6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b2ddff853b7546788b645a0344c2297b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:29:52.797840 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-e6faddcf-ecf2-45ad-a7d6-d4634b9603d0 tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] 10.222.0.52 "GET /compute/v2.1/servers/9941c49f-8626-461d-a570-c0ffbfd4b804" status: 200 len: 1569 microversion: 2.1 time: 0.279694 Aug 27 02:29:52.798422 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 370/736] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:29:52 2022] GET /compute/v2.1/servers/9941c49f-8626-461d-a570-c0ffbfd4b804 => generated 1569 bytes in 281 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:29:53.007495 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-6cc3c440-ee49-4f0f-9e22-71b1a8a98d45 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:29:53.008888 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-6cc3c440-ee49-4f0f-9e22-71b1a8a98d45 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] [instance: dc699df0-be4f-4036-9e01-45ba0156616b] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:29:53.016589 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-6cc3c440-ee49-4f0f-9e22-71b1a8a98d45 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:29:53.017049 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-6cc3c440-ee49-4f0f-9e22-71b1a8a98d45 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:29:53.017508 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-6cc3c440-ee49-4f0f-9e22-71b1a8a98d45 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:29:53.018217 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-6cc3c440-ee49-4f0f-9e22-71b1a8a98d45 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:29:53.018667 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-6cc3c440-ee49-4f0f-9e22-71b1a8a98d45 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:29:53.019301 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-6cc3c440-ee49-4f0f-9e22-71b1a8a98d45 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:29:53.306635 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-6cc3c440-ee49-4f0f-9e22-71b1a8a98d45 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '26a86351583d49d090b3db09f127d49f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5d5ec8450c0941d18d63558baab54737', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:29:53.314722 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-6cc3c440-ee49-4f0f-9e22-71b1a8a98d45 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] 10.222.0.52 "GET /compute/v2.1/servers/dc699df0-be4f-4036-9e01-45ba0156616b" status: 200 len: 1680 microversion: 2.1 time: 0.311917 Aug 27 02:29:53.315319 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 367/737] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:29:53 2022] GET /compute/v2.1/servers/dc699df0-be4f-4036-9e01-45ba0156616b => generated 1680 bytes in 313 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:29:53.361489 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-662747ce-de5e-49b1-9e6a-caed6a7bc741 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Calling method '>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:29:53.362025 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-662747ce-de5e-49b1-9e6a-caed6a7bc741 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] [instance: 98e4cfeb-d49e-4888-a2c0-40a9e8af90d3] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:29:53.369005 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-662747ce-de5e-49b1-9e6a-caed6a7bc741 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:29:53.369530 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-662747ce-de5e-49b1-9e6a-caed6a7bc741 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:29:53.369963 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-662747ce-de5e-49b1-9e6a-caed6a7bc741 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:29:53.370922 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-662747ce-de5e-49b1-9e6a-caed6a7bc741 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:29:53.371451 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-662747ce-de5e-49b1-9e6a-caed6a7bc741 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:29:53.371943 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-662747ce-de5e-49b1-9e6a-caed6a7bc741 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:29:53.590354 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-662747ce-de5e-49b1-9e6a-caed6a7bc741 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd945d512ae034ec08838d0a3afde2429', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '880ac4003248441c809a76cfd1b221eb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:29:53.599634 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-662747ce-de5e-49b1-9e6a-caed6a7bc741 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] 10.222.0.52 "GET /compute/v2.1/servers/98e4cfeb-d49e-4888-a2c0-40a9e8af90d3" status: 200 len: 1735 microversion: 2.1 time: 0.242074 Aug 27 02:29:53.600587 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 371/738] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:29:53 2022] GET /compute/v2.1/servers/98e4cfeb-d49e-4888-a2c0-40a9e8af90d3 => generated 1735 bytes in 244 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:29:53.826358 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-75bae3f6-1891-40b9-8f2a-51ac7d081915 tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:29:53.827083 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-75bae3f6-1891-40b9-8f2a-51ac7d081915 tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] [instance: 9941c49f-8626-461d-a570-c0ffbfd4b804] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:29:53.833642 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-75bae3f6-1891-40b9-8f2a-51ac7d081915 tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:29:53.834231 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-75bae3f6-1891-40b9-8f2a-51ac7d081915 tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:29:53.834730 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-75bae3f6-1891-40b9-8f2a-51ac7d081915 tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:29:53.835924 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-75bae3f6-1891-40b9-8f2a-51ac7d081915 tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:29:53.836719 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-75bae3f6-1891-40b9-8f2a-51ac7d081915 tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:29:53.837232 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-75bae3f6-1891-40b9-8f2a-51ac7d081915 tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:29:54.015296 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [req-5fd8763a-a1ad-43aa-8651-0d38a9bfc05f req-322943f0-8e0d-48cb-9489-d721e9da0627 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "dc699df0-be4f-4036-9e01-45ba0156616b", "name": "network-vif-deleted", "tag": "2fe1c73f-fd8a-4318-9640-cf25059ea1c3"}]} {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 27 02:29:54.026696 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [req-5fd8763a-a1ad-43aa-8651-0d38a9bfc05f req-322943f0-8e0d-48cb-9489-d721e9da0627 service nova] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:29:54.027472 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [req-5fd8763a-a1ad-43aa-8651-0d38a9bfc05f req-322943f0-8e0d-48cb-9489-d721e9da0627 service nova] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:29:54.028071 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [req-5fd8763a-a1ad-43aa-8651-0d38a9bfc05f req-322943f0-8e0d-48cb-9489-d721e9da0627 service nova] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:29:54.048880 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.compute.server_external_events [req-5fd8763a-a1ad-43aa-8651-0d38a9bfc05f req-322943f0-8e0d-48cb-9489-d721e9da0627 service nova] Creating event network-vif-deleted:2fe1c73f-fd8a-4318-9640-cf25059ea1c3 for instance dc699df0-be4f-4036-9e01-45ba0156616b on hv-cinder-84630 Aug 27 02:29:54.055125 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [req-5fd8763a-a1ad-43aa-8651-0d38a9bfc05f req-322943f0-8e0d-48cb-9489-d721e9da0627 service nova] 10.222.0.52 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.100600 Aug 27 02:29:54.055784 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-75bae3f6-1891-40b9-8f2a-51ac7d081915 tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '642291c4daa04a6ab3fbce92a8be39c6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b2ddff853b7546788b645a0344c2297b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:29:54.056291 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 372/739] 10.222.0.52 () {64 vars in 1317 bytes} [Sat Aug 27 02:29:53 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 103 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 27 02:29:54.065815 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-75bae3f6-1891-40b9-8f2a-51ac7d081915 tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] 10.222.0.52 "GET /compute/v2.1/servers/9941c49f-8626-461d-a570-c0ffbfd4b804" status: 200 len: 1569 microversion: 2.1 time: 0.247879 Aug 27 02:29:54.066398 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 368/740] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:29:53 2022] GET /compute/v2.1/servers/9941c49f-8626-461d-a570-c0ffbfd4b804 => generated 1569 bytes in 250 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:29:54.339589 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-49f26788-bc9f-4c00-83e7-c1542a9bdb1b tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Calling method '>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:29:54.340685 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-49f26788-bc9f-4c00-83e7-c1542a9bdb1b tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] [instance: dc699df0-be4f-4036-9e01-45ba0156616b] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:29:54.352903 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-49f26788-bc9f-4c00-83e7-c1542a9bdb1b tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:29:54.353807 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-49f26788-bc9f-4c00-83e7-c1542a9bdb1b tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:29:54.354486 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-49f26788-bc9f-4c00-83e7-c1542a9bdb1b tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:29:54.355459 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-49f26788-bc9f-4c00-83e7-c1542a9bdb1b tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:29:54.356158 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-49f26788-bc9f-4c00-83e7-c1542a9bdb1b tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:29:54.356796 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-49f26788-bc9f-4c00-83e7-c1542a9bdb1b tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:29:54.432489 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-49f26788-bc9f-4c00-83e7-c1542a9bdb1b tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '26a86351583d49d090b3db09f127d49f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5d5ec8450c0941d18d63558baab54737', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:29:54.449639 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-49f26788-bc9f-4c00-83e7-c1542a9bdb1b tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] 10.222.0.52 "GET /compute/v2.1/servers/dc699df0-be4f-4036-9e01-45ba0156616b" status: 200 len: 1613 microversion: 2.1 time: 0.115509 Aug 27 02:29:54.450874 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 373/741] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:29:54 2022] GET /compute/v2.1/servers/dc699df0-be4f-4036-9e01-45ba0156616b => generated 1613 bytes in 118 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:29:54.620968 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-177b213e-1386-46b3-89fe-118afc8e89af tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:29:54.621662 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-177b213e-1386-46b3-89fe-118afc8e89af tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] [instance: 98e4cfeb-d49e-4888-a2c0-40a9e8af90d3] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:29:54.627569 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-177b213e-1386-46b3-89fe-118afc8e89af tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:29:54.628058 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-177b213e-1386-46b3-89fe-118afc8e89af tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:29:54.628704 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-177b213e-1386-46b3-89fe-118afc8e89af tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:29:54.629453 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-177b213e-1386-46b3-89fe-118afc8e89af tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:29:54.629918 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-177b213e-1386-46b3-89fe-118afc8e89af tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:29:54.630341 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-177b213e-1386-46b3-89fe-118afc8e89af tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:29:54.801331 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-177b213e-1386-46b3-89fe-118afc8e89af tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd945d512ae034ec08838d0a3afde2429', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '880ac4003248441c809a76cfd1b221eb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:29:54.812432 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-177b213e-1386-46b3-89fe-118afc8e89af tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] 10.222.0.52 "GET /compute/v2.1/servers/98e4cfeb-d49e-4888-a2c0-40a9e8af90d3" status: 200 len: 1735 microversion: 2.1 time: 0.195504 Aug 27 02:29:54.813013 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 369/742] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:29:54 2022] GET /compute/v2.1/servers/98e4cfeb-d49e-4888-a2c0-40a9e8af90d3 => generated 1735 bytes in 197 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:29:55.090287 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-e4cdf54c-daae-4bfb-82ee-c35c530e34c1 tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] Calling method '>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:29:55.091704 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-e4cdf54c-daae-4bfb-82ee-c35c530e34c1 tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] [instance: 9941c49f-8626-461d-a570-c0ffbfd4b804] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:29:55.100381 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-e4cdf54c-daae-4bfb-82ee-c35c530e34c1 tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:29:55.100871 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-e4cdf54c-daae-4bfb-82ee-c35c530e34c1 tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:29:55.101886 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-e4cdf54c-daae-4bfb-82ee-c35c530e34c1 tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:29:55.102320 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-e4cdf54c-daae-4bfb-82ee-c35c530e34c1 tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:29:55.102948 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-e4cdf54c-daae-4bfb-82ee-c35c530e34c1 tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:29:55.103541 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-e4cdf54c-daae-4bfb-82ee-c35c530e34c1 tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:29:55.122430 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG dbcounter [-] [120864] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=120864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 27 02:29:55.267472 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-e4cdf54c-daae-4bfb-82ee-c35c530e34c1 tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '642291c4daa04a6ab3fbce92a8be39c6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b2ddff853b7546788b645a0344c2297b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:29:55.284781 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-e4cdf54c-daae-4bfb-82ee-c35c530e34c1 tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] 10.222.0.52 "GET /compute/v2.1/servers/9941c49f-8626-461d-a570-c0ffbfd4b804" status: 200 len: 1569 microversion: 2.1 time: 0.200706 Aug 27 02:29:55.295344 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 374/743] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:29:55 2022] GET /compute/v2.1/servers/9941c49f-8626-461d-a570-c0ffbfd4b804 => generated 1569 bytes in 212 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:29:55.488603 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-a004a491-57ff-4381-93af-665f296d27e3 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:29:55.489466 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-a004a491-57ff-4381-93af-665f296d27e3 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] [instance: dc699df0-be4f-4036-9e01-45ba0156616b] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:29:55.499727 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-a004a491-57ff-4381-93af-665f296d27e3 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:29:55.500672 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-a004a491-57ff-4381-93af-665f296d27e3 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:29:55.501660 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-a004a491-57ff-4381-93af-665f296d27e3 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:29:55.502733 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-a004a491-57ff-4381-93af-665f296d27e3 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:29:55.503476 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-a004a491-57ff-4381-93af-665f296d27e3 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:29:55.504464 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-a004a491-57ff-4381-93af-665f296d27e3 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:29:55.552144 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-a004a491-57ff-4381-93af-665f296d27e3 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:29:55.552543 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-a004a491-57ff-4381-93af-665f296d27e3 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:29:55.553239 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-a004a491-57ff-4381-93af-665f296d27e3 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:29:55.553840 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.objects.instance [None req-a004a491-57ff-4381-93af-665f296d27e3 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lazy-loading 'fault' on Instance uuid dc699df0-be4f-4036-9e01-45ba0156616b {{(pid=120863) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Aug 27 02:29:55.607200 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-a004a491-57ff-4381-93af-665f296d27e3 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '26a86351583d49d090b3db09f127d49f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5d5ec8450c0941d18d63558baab54737', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:29:55.615065 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-a004a491-57ff-4381-93af-665f296d27e3 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] 10.222.0.52 "GET /compute/v2.1/servers/dc699df0-be4f-4036-9e01-45ba0156616b" status: 200 len: 1331 microversion: 2.1 time: 0.140033 Aug 27 02:29:55.621084 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 370/744] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:29:55 2022] GET /compute/v2.1/servers/dc699df0-be4f-4036-9e01-45ba0156616b => generated 1331 bytes in 141 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:29:55.838205 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-32cc364c-3b34-4ca5-bbc6-228f0d3cae26 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Calling method '>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:29:55.838806 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-32cc364c-3b34-4ca5-bbc6-228f0d3cae26 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] [instance: 98e4cfeb-d49e-4888-a2c0-40a9e8af90d3] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:29:55.844771 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-32cc364c-3b34-4ca5-bbc6-228f0d3cae26 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:29:55.845302 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-32cc364c-3b34-4ca5-bbc6-228f0d3cae26 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:29:55.845730 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-32cc364c-3b34-4ca5-bbc6-228f0d3cae26 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:29:55.846422 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-32cc364c-3b34-4ca5-bbc6-228f0d3cae26 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:29:55.847057 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-32cc364c-3b34-4ca5-bbc6-228f0d3cae26 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:29:55.847473 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-32cc364c-3b34-4ca5-bbc6-228f0d3cae26 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:29:55.979098 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-32cc364c-3b34-4ca5-bbc6-228f0d3cae26 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd945d512ae034ec08838d0a3afde2429', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '880ac4003248441c809a76cfd1b221eb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:29:55.988112 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-32cc364c-3b34-4ca5-bbc6-228f0d3cae26 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] 10.222.0.52 "GET /compute/v2.1/servers/98e4cfeb-d49e-4888-a2c0-40a9e8af90d3" status: 200 len: 1735 microversion: 2.1 time: 0.154050 Aug 27 02:29:55.988699 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 375/745] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:29:55 2022] GET /compute/v2.1/servers/98e4cfeb-d49e-4888-a2c0-40a9e8af90d3 => generated 1735 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:29:56.311194 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-35d64d1b-1d34-41e4-b155-e04ab0196f0b tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:29:56.311961 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-35d64d1b-1d34-41e4-b155-e04ab0196f0b tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] [instance: 9941c49f-8626-461d-a570-c0ffbfd4b804] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:29:56.319514 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-35d64d1b-1d34-41e4-b155-e04ab0196f0b tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:29:56.319840 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-35d64d1b-1d34-41e4-b155-e04ab0196f0b tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:29:56.320374 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-35d64d1b-1d34-41e4-b155-e04ab0196f0b tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:29:56.321282 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-35d64d1b-1d34-41e4-b155-e04ab0196f0b tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:29:56.321631 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-35d64d1b-1d34-41e4-b155-e04ab0196f0b tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:29:56.322052 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-35d64d1b-1d34-41e4-b155-e04ab0196f0b tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:29:56.468925 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-35d64d1b-1d34-41e4-b155-e04ab0196f0b tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '642291c4daa04a6ab3fbce92a8be39c6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b2ddff853b7546788b645a0344c2297b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:29:56.477763 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-35d64d1b-1d34-41e4-b155-e04ab0196f0b tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] 10.222.0.52 "GET /compute/v2.1/servers/9941c49f-8626-461d-a570-c0ffbfd4b804" status: 200 len: 1510 microversion: 2.1 time: 0.170793 Aug 27 02:29:56.478365 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 371/746] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:29:56 2022] GET /compute/v2.1/servers/9941c49f-8626-461d-a570-c0ffbfd4b804 => generated 1510 bytes in 172 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:29:56.499143 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-e229b80f-734f-4992-b889-76e9d9c0c8cf tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] Action: 'action', calling method: >, body: {"unshelve": null} {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 27 02:29:56.499762 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-e229b80f-734f-4992-b889-76e9d9c0c8cf tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] [instance: 9941c49f-8626-461d-a570-c0ffbfd4b804] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:29:56.506395 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-e229b80f-734f-4992-b889-76e9d9c0c8cf tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:29:56.506874 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-e229b80f-734f-4992-b889-76e9d9c0c8cf tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:29:56.507303 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-e229b80f-734f-4992-b889-76e9d9c0c8cf tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:29:56.508033 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-e229b80f-734f-4992-b889-76e9d9c0c8cf tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:29:56.508476 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-e229b80f-734f-4992-b889-76e9d9c0c8cf tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:29:56.508887 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-e229b80f-734f-4992-b889-76e9d9c0c8cf tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:29:56.539551 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-e229b80f-734f-4992-b889-76e9d9c0c8cf tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] [instance: 9941c49f-8626-461d-a570-c0ffbfd4b804] Unshelving instance with old availability_zone "None" to new availability_zone "not provided" and host "None". {{(pid=120864) unshelve /opt/stack/nova/nova/compute/api.py:4591}} Aug 27 02:29:56.608264 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-e229b80f-734f-4992-b889-76e9d9c0c8cf tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] 10.222.0.52 "POST /compute/v2.1/servers/9941c49f-8626-461d-a570-c0ffbfd4b804/action" status: 202 len: 0 microversion: 2.1 time: 0.113123 Aug 27 02:29:56.608896 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 376/747] 10.222.0.52 () {60 vars in 1278 bytes} [Sat Aug 27 02:29:56 2022] POST /compute/v2.1/servers/9941c49f-8626-461d-a570-c0ffbfd4b804/action => generated 0 bytes in 114 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Aug 27 02:29:56.616918 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-7d7643da-5cf1-445a-8e71-590b13f29f7f tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:29:56.617752 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-7d7643da-5cf1-445a-8e71-590b13f29f7f tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] [instance: 9941c49f-8626-461d-a570-c0ffbfd4b804] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:29:56.624452 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-7d7643da-5cf1-445a-8e71-590b13f29f7f tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:29:56.624948 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-7d7643da-5cf1-445a-8e71-590b13f29f7f tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:29:56.625409 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-7d7643da-5cf1-445a-8e71-590b13f29f7f tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:29:56.626787 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-7d7643da-5cf1-445a-8e71-590b13f29f7f tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:29:56.626787 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-7d7643da-5cf1-445a-8e71-590b13f29f7f tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:29:56.626985 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-7d7643da-5cf1-445a-8e71-590b13f29f7f tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:29:56.637236 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-cd05de0b-7cf7-4321-8458-ed249c8dc6a6 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Calling method '>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:29:56.638084 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-cd05de0b-7cf7-4321-8458-ed249c8dc6a6 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] [instance: dc699df0-be4f-4036-9e01-45ba0156616b] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:29:56.644185 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-cd05de0b-7cf7-4321-8458-ed249c8dc6a6 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:29:56.644900 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-cd05de0b-7cf7-4321-8458-ed249c8dc6a6 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:29:56.645354 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-cd05de0b-7cf7-4321-8458-ed249c8dc6a6 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:29:56.646043 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-cd05de0b-7cf7-4321-8458-ed249c8dc6a6 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:29:56.646507 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-cd05de0b-7cf7-4321-8458-ed249c8dc6a6 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:29:56.647056 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-cd05de0b-7cf7-4321-8458-ed249c8dc6a6 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:29:56.665577 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.wsgi [None req-cd05de0b-7cf7-4321-8458-ed249c8dc6a6 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] HTTP exception thrown: Instance dc699df0-be4f-4036-9e01-45ba0156616b could not be found. Aug 27 02:29:56.667254 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-cd05de0b-7cf7-4321-8458-ed249c8dc6a6 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Request method failure captured: Aug 27 02:29:56.667254 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: request: GET /compute/v2.1/servers/dc699df0-be4f-4036-9e01-45ba0156616b HTTP/1.1 Aug 27 02:29:56.667254 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: Accept: application/json Aug 27 02:29:56.667254 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: Accept-Encoding: identity Aug 27 02:29:56.667254 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: Connection: close Aug 27 02:29:56.667254 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: Content-Length: 0 Aug 27 02:29:56.667254 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: Content-Type: application/json Aug 27 02:29:56.667254 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: Host: 10.222.0.52 Aug 27 02:29:56.667254 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: Openstack-System-Scope: None Aug 27 02:29:56.667254 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: User-Agent: python-urllib3/1.26.11 Aug 27 02:29:56.667254 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-Auth-Token: gAAAAABjCX_pkOb4-Fffq6Hbe0cfjlj1ARL1JmORSzrYWucWmWgZ67LEorDD3IWpPkj3UP534US8GW_nja0iAKwcdxSx3e5teQzZMdrCvOvA5aSGJO8iodOjhfOIsl8L0d8om8kj8_3q-8uTHxKupSTnIQy_J-f1kC4e22VNejIIiExDsGitats Aug 27 02:29:56.667254 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-Domain-Id: None Aug 27 02:29:56.667254 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-Domain-Name: None Aug 27 02:29:56.667254 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-Identity-Status: Confirmed Aug 27 02:29:56.667254 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-Is-Admin-Project: True Aug 27 02:29:56.667254 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-Project-Domain-Id: default Aug 27 02:29:56.667254 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-Project-Domain-Name: Default Aug 27 02:29:56.667254 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-Project-Id: 5d5ec8450c0941d18d63558baab54737 Aug 27 02:29:56.667254 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-Project-Name: tempest-AttachVolumeNegativeTest-1988669364 Aug 27 02:29:56.667254 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-Role: reader,member Aug 27 02:29:56.667254 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-Roles: reader,member Aug 27 02:29:56.668913 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-Service-Catalog: [{"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.52:8080", "publicURL": "http://10.222.0.52:8080/v1/AUTH_5d5ec8450c0941d18d63558baab54737"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52:9696/networking"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/compute/v2/5d5ec8450c0941d18d63558baab54737"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/image"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/volume/v3/5d5ec8450c0941d18d63558baab54737"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/identity"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/volume/v3/5d5ec8450c0941d18d63558baab54737"}]}] Aug 27 02:29:56.668913 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-Tenant: tempest-AttachVolumeNegativeTest-1988669364 Aug 27 02:29:56.668913 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-Tenant-Id: 5d5ec8450c0941d18d63558baab54737 Aug 27 02:29:56.668913 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-1988669364 Aug 27 02:29:56.668913 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-User: tempest-AttachVolumeNegativeTest-1988669364-project Aug 27 02:29:56.668913 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-User-Domain-Id: default Aug 27 02:29:56.668913 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-User-Domain-Name: Default Aug 27 02:29:56.668913 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-User-Id: 26a86351583d49d090b3db09f127d49f Aug 27 02:29:56.668913 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-User-Name: tempest-AttachVolumeNegativeTest-1988669364-project Aug 27 02:29:56.668913 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: method: > Aug 27 02:29:56.668913 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: exception: Instance dc699df0-be4f-4036-9e01-45ba0156616b could not be found. Aug 27 02:29:56.668913 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: action_args: {'id': 'dc699df0-be4f-4036-9e01-45ba0156616b'} Aug 27 02:29:56.670108 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Aug 27 02:29:56.670108 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 27 02:29:56.670108 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Aug 27 02:29:56.670108 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Aug 27 02:29:56.670108 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Aug 27 02:29:56.670108 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Aug 27 02:29:56.670108 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Aug 27 02:29:56.670108 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Aug 27 02:29:56.670108 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Aug 27 02:29:56.670108 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Aug 27 02:29:56.670108 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance dc699df0-be4f-4036-9e01-45ba0156616b could not be found. Aug 27 02:29:56.670108 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi Aug 27 02:29:56.670108 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 27 02:29:56.670108 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi Aug 27 02:29:56.670108 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 27 02:29:56.670108 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 27 02:29:56.670108 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 27 02:29:56.670108 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Aug 27 02:29:56.670108 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Aug 27 02:29:56.670108 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Aug 27 02:29:56.671530 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Aug 27 02:29:56.671530 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Aug 27 02:29:56.671530 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi instance = self._get_server( Aug 27 02:29:56.671530 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Aug 27 02:29:56.671530 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Aug 27 02:29:56.671530 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Aug 27 02:29:56.671530 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Aug 27 02:29:56.671530 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance dc699df0-be4f-4036-9e01-45ba0156616b could not be found. Aug 27 02:29:56.671530 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi Aug 27 02:29:56.671530 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 27 02:29:56.671530 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi Aug 27 02:29:56.671530 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 27 02:29:56.671530 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 27 02:29:56.671530 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 27 02:29:56.671530 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Aug 27 02:29:56.671530 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Aug 27 02:29:56.671530 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance dc699df0-be4f-4036-9e01-45ba0156616b could not be found. Aug 27 02:29:56.671530 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi Aug 27 02:29:56.673029 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-cd05de0b-7cf7-4321-8458-ed249c8dc6a6 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Returning 404 to user: Instance dc699df0-be4f-4036-9e01-45ba0156616b could not be found. {{(pid=120864) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Aug 27 02:29:56.673029 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-cd05de0b-7cf7-4321-8458-ed249c8dc6a6 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] 10.222.0.52 "GET /compute/v2.1/servers/dc699df0-be4f-4036-9e01-45ba0156616b" status: 404 len: 111 microversion: 2.1 time: 0.036929 Aug 27 02:29:56.673029 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 377/748] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:29:56 2022] GET /compute/v2.1/servers/dc699df0-be4f-4036-9e01-45ba0156616b => generated 111 bytes in 41 msecs (HTTP/1.1 404) 9 headers in 379 bytes (2 switches on core 0) Aug 27 02:29:56.677930 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-fe87d5ad-1301-42fd-8156-14a056d434e9 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Calling method '.version_select at 0x7f6b1b2a4940>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:29:56.702148 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-fe87d5ad-1301-42fd-8156-14a056d434e9 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] 10.222.0.52 "DELETE /compute/v2.1/os-keypairs/tempest-keypair-383499095" status: 202 len: 0 microversion: 2.1 time: 0.027123 Aug 27 02:29:56.703059 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 378/749] 10.222.0.52 () {58 vars in 1218 bytes} [Sat Aug 27 02:29:56 2022] DELETE /compute/v2.1/os-keypairs/tempest-keypair-383499095 => generated 0 bytes in 29 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Aug 27 02:29:56.806465 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-7d7643da-5cf1-445a-8e71-590b13f29f7f tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '642291c4daa04a6ab3fbce92a8be39c6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b2ddff853b7546788b645a0344c2297b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:29:56.808728 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG dbcounter [-] [120863] Writing DB stats nova_cell1:SELECT=100 {{(pid=120863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 27 02:29:56.816006 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-7d7643da-5cf1-445a-8e71-590b13f29f7f tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] 10.222.0.52 "GET /compute/v2.1/servers/9941c49f-8626-461d-a570-c0ffbfd4b804" status: 200 len: 1518 microversion: 2.1 time: 0.202799 Aug 27 02:29:56.816640 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 372/750] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:29:56 2022] GET /compute/v2.1/servers/9941c49f-8626-461d-a570-c0ffbfd4b804 => generated 1518 bytes in 204 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:29:57.010321 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-6b6d4408-cf72-4e41-8aed-3d40c6ddc96d tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Calling method '>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:29:57.011050 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-6b6d4408-cf72-4e41-8aed-3d40c6ddc96d tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] [instance: 98e4cfeb-d49e-4888-a2c0-40a9e8af90d3] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:29:57.018136 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-6b6d4408-cf72-4e41-8aed-3d40c6ddc96d tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:29:57.018624 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-6b6d4408-cf72-4e41-8aed-3d40c6ddc96d tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:29:57.019058 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-6b6d4408-cf72-4e41-8aed-3d40c6ddc96d tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:29:57.019762 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-6b6d4408-cf72-4e41-8aed-3d40c6ddc96d tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:29:57.020212 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-6b6d4408-cf72-4e41-8aed-3d40c6ddc96d tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:29:57.020634 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-6b6d4408-cf72-4e41-8aed-3d40c6ddc96d tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:29:57.195906 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-6b6d4408-cf72-4e41-8aed-3d40c6ddc96d tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd945d512ae034ec08838d0a3afde2429', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '880ac4003248441c809a76cfd1b221eb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:29:57.206302 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-6b6d4408-cf72-4e41-8aed-3d40c6ddc96d tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] 10.222.0.52 "GET /compute/v2.1/servers/98e4cfeb-d49e-4888-a2c0-40a9e8af90d3" status: 200 len: 1735 microversion: 2.1 time: 0.199821 Aug 27 02:29:57.206560 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 379/751] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:29:57 2022] GET /compute/v2.1/servers/98e4cfeb-d49e-4888-a2c0-40a9e8af90d3 => generated 1735 bytes in 201 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:29:57.554236 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-c4470e19-9b20-474d-825f-c6b88bf75e10 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:29:57.555710 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-c4470e19-9b20-474d-825f-c6b88bf75e10 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] [instance: 2ad09ab6-ef1f-47a2-bf44-6ae046cc8645] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:29:57.564110 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-c4470e19-9b20-474d-825f-c6b88bf75e10 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:29:57.564854 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-c4470e19-9b20-474d-825f-c6b88bf75e10 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:29:57.565349 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-c4470e19-9b20-474d-825f-c6b88bf75e10 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:29:57.566058 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-c4470e19-9b20-474d-825f-c6b88bf75e10 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:29:57.566578 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-c4470e19-9b20-474d-825f-c6b88bf75e10 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:29:57.567034 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-c4470e19-9b20-474d-825f-c6b88bf75e10 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:29:57.682971 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.objects.instance [None req-c4470e19-9b20-474d-825f-c6b88bf75e10 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lazy-loading 'services' on Instance uuid 2ad09ab6-ef1f-47a2-bf44-6ae046cc8645 {{(pid=120863) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Aug 27 02:29:57.778694 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-c4470e19-9b20-474d-825f-c6b88bf75e10 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] 10.222.0.52 "DELETE /compute/v2.1/servers/2ad09ab6-ef1f-47a2-bf44-6ae046cc8645/os-volume_attachments/58ee6df8-0ef2-453b-82ec-e4643432a653" status: 202 len: 0 microversion: 2.1 time: 0.230084 Aug 27 02:29:57.779305 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 373/752] 10.222.0.52 () {58 vars in 1416 bytes} [Sat Aug 27 02:29:57 2022] DELETE /compute/v2.1/servers/2ad09ab6-ef1f-47a2-bf44-6ae046cc8645/os-volume_attachments/58ee6df8-0ef2-453b-82ec-e4643432a653 => generated 0 bytes in 232 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Aug 27 02:29:57.791897 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-bbf38c86-6922-48df-a552-a5acce2dd83d tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Calling method '>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:29:57.795805 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-bbf38c86-6922-48df-a552-a5acce2dd83d tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] [instance: 2ad09ab6-ef1f-47a2-bf44-6ae046cc8645] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:29:57.804330 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-bbf38c86-6922-48df-a552-a5acce2dd83d tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:29:57.805314 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-bbf38c86-6922-48df-a552-a5acce2dd83d tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:29:57.805948 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-bbf38c86-6922-48df-a552-a5acce2dd83d tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:29:57.806654 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-bbf38c86-6922-48df-a552-a5acce2dd83d tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:29:57.807227 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-bbf38c86-6922-48df-a552-a5acce2dd83d tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:29:57.807935 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-bbf38c86-6922-48df-a552-a5acce2dd83d tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:29:57.840482 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-70902412-9938-4e45-ba5a-bcec116a713d tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:29:57.841482 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-70902412-9938-4e45-ba5a-bcec116a713d tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] [instance: 9941c49f-8626-461d-a570-c0ffbfd4b804] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:29:57.842929 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-bbf38c86-6922-48df-a552-a5acce2dd83d tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] 10.222.0.52 "GET /compute/v2.1/servers/2ad09ab6-ef1f-47a2-bf44-6ae046cc8645/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.058503 Aug 27 02:29:57.847634 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 380/753] 10.222.0.52 () {58 vars in 1302 bytes} [Sat Aug 27 02:29:57 2022] GET /compute/v2.1/servers/2ad09ab6-ef1f-47a2-bf44-6ae046cc8645/os-volume_attachments => generated 197 bytes in 60 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 27 02:29:57.848978 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-70902412-9938-4e45-ba5a-bcec116a713d tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:29:57.849518 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-70902412-9938-4e45-ba5a-bcec116a713d tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:29:57.849959 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-70902412-9938-4e45-ba5a-bcec116a713d tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:29:57.851345 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-70902412-9938-4e45-ba5a-bcec116a713d tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:29:57.852015 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-70902412-9938-4e45-ba5a-bcec116a713d tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:29:57.852511 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-70902412-9938-4e45-ba5a-bcec116a713d tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:29:57.907295 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [req-ac240d5c-4e14-4c89-bb6c-4ce269abba67 req-6a18d11f-22de-44b4-990e-f2a735d5564b service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "9941c49f-8626-461d-a570-c0ffbfd4b804", "name": "network-vif-unplugged", "status": "completed", "tag": "89f4d982-1807-4e80-b9ef-c7bc0025f1b0"}]} {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 27 02:29:57.919246 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [req-ac240d5c-4e14-4c89-bb6c-4ce269abba67 req-6a18d11f-22de-44b4-990e-f2a735d5564b service nova] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:29:57.920094 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [req-ac240d5c-4e14-4c89-bb6c-4ce269abba67 req-6a18d11f-22de-44b4-990e-f2a735d5564b service nova] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:29:57.920695 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [req-ac240d5c-4e14-4c89-bb6c-4ce269abba67 req-6a18d11f-22de-44b4-990e-f2a735d5564b service nova] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:29:57.939471 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.compute.server_external_events [req-ac240d5c-4e14-4c89-bb6c-4ce269abba67 req-6a18d11f-22de-44b4-990e-f2a735d5564b service nova] Unable to find a host for instance 9941c49f-8626-461d-a570-c0ffbfd4b804. Dropping event network-vif-unplugged {{(pid=120864) create /opt/stack/nova/nova/api/openstack/compute/server_external_events.py:143}} Aug 27 02:29:57.941574 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [req-ac240d5c-4e14-4c89-bb6c-4ce269abba67 req-6a18d11f-22de-44b4-990e-f2a735d5564b service nova] 10.222.0.52 "POST /compute/v2.1/os-server-external-events" status: 207 len: 182 microversion: 2.1 time: 0.042667 Aug 27 02:29:57.942754 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 381/754] 10.222.0.52 () {64 vars in 1317 bytes} [Sat Aug 27 02:29:57 2022] POST /compute/v2.1/os-server-external-events => generated 182 bytes in 46 msecs (HTTP/1.1 207) 9 headers in 367 bytes (1 switches on core 0) Aug 27 02:29:58.037090 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-70902412-9938-4e45-ba5a-bcec116a713d tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '642291c4daa04a6ab3fbce92a8be39c6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b2ddff853b7546788b645a0344c2297b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:29:58.045404 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-70902412-9938-4e45-ba5a-bcec116a713d tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] 10.222.0.52 "GET /compute/v2.1/servers/9941c49f-8626-461d-a570-c0ffbfd4b804" status: 200 len: 1520 microversion: 2.1 time: 0.211597 Aug 27 02:29:58.045998 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 374/755] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:29:57 2022] GET /compute/v2.1/servers/9941c49f-8626-461d-a570-c0ffbfd4b804 => generated 1520 bytes in 213 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 27 02:29:58.228665 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-6c202615-1f90-4fc8-aa3a-0870a76b6b02 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Calling method '>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:29:58.229215 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-6c202615-1f90-4fc8-aa3a-0870a76b6b02 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] [instance: 98e4cfeb-d49e-4888-a2c0-40a9e8af90d3] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:29:58.235290 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-6c202615-1f90-4fc8-aa3a-0870a76b6b02 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:29:58.235756 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-6c202615-1f90-4fc8-aa3a-0870a76b6b02 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:29:58.236182 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-6c202615-1f90-4fc8-aa3a-0870a76b6b02 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:29:58.236856 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-6c202615-1f90-4fc8-aa3a-0870a76b6b02 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:29:58.237318 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-6c202615-1f90-4fc8-aa3a-0870a76b6b02 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:29:58.237732 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-6c202615-1f90-4fc8-aa3a-0870a76b6b02 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:29:58.411482 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-6c202615-1f90-4fc8-aa3a-0870a76b6b02 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd945d512ae034ec08838d0a3afde2429', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '880ac4003248441c809a76cfd1b221eb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:29:58.427850 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-6c202615-1f90-4fc8-aa3a-0870a76b6b02 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] 10.222.0.52 "GET /compute/v2.1/servers/98e4cfeb-d49e-4888-a2c0-40a9e8af90d3" status: 200 len: 1735 microversion: 2.1 time: 0.202904 Aug 27 02:29:58.428928 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 382/756] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:29:58 2022] GET /compute/v2.1/servers/98e4cfeb-d49e-4888-a2c0-40a9e8af90d3 => generated 1735 bytes in 205 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 27 02:29:58.862074 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-8427fb02-770d-4149-8195-b2ff08374dfa tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:29:58.865935 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-8427fb02-770d-4149-8195-b2ff08374dfa tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] [instance: 2ad09ab6-ef1f-47a2-bf44-6ae046cc8645] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:29:58.873696 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-8427fb02-770d-4149-8195-b2ff08374dfa tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:29:58.874191 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-8427fb02-770d-4149-8195-b2ff08374dfa tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:29:58.874630 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-8427fb02-770d-4149-8195-b2ff08374dfa tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:29:58.875306 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-8427fb02-770d-4149-8195-b2ff08374dfa tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:29:58.875762 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-8427fb02-770d-4149-8195-b2ff08374dfa tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:29:58.876187 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-8427fb02-770d-4149-8195-b2ff08374dfa tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:29:58.910821 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-8427fb02-770d-4149-8195-b2ff08374dfa tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] 10.222.0.52 "GET /compute/v2.1/servers/2ad09ab6-ef1f-47a2-bf44-6ae046cc8645/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.056692 Aug 27 02:29:58.911395 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 375/757] 10.222.0.52 () {58 vars in 1302 bytes} [Sat Aug 27 02:29:58 2022] GET /compute/v2.1/servers/2ad09ab6-ef1f-47a2-bf44-6ae046cc8645/os-volume_attachments => generated 197 bytes in 59 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Aug 27 02:29:59.068623 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-ffe25923-25c0-42c3-8976-ecb80e696337 tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] Calling method '>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:29:59.069259 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-ffe25923-25c0-42c3-8976-ecb80e696337 tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] [instance: 9941c49f-8626-461d-a570-c0ffbfd4b804] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:29:59.076058 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-ffe25923-25c0-42c3-8976-ecb80e696337 tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:29:59.076709 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-ffe25923-25c0-42c3-8976-ecb80e696337 tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:29:59.077213 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-ffe25923-25c0-42c3-8976-ecb80e696337 tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:29:59.077903 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-ffe25923-25c0-42c3-8976-ecb80e696337 tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:29:59.078346 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-ffe25923-25c0-42c3-8976-ecb80e696337 tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:29:59.078898 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-ffe25923-25c0-42c3-8976-ecb80e696337 tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:29:59.234948 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-ffe25923-25c0-42c3-8976-ecb80e696337 tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '642291c4daa04a6ab3fbce92a8be39c6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b2ddff853b7546788b645a0344c2297b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:29:59.243631 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-ffe25923-25c0-42c3-8976-ecb80e696337 tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] 10.222.0.52 "GET /compute/v2.1/servers/9941c49f-8626-461d-a570-c0ffbfd4b804" status: 200 len: 1576 microversion: 2.1 time: 0.179578 Aug 27 02:29:59.244227 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 383/758] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:29:59 2022] GET /compute/v2.1/servers/9941c49f-8626-461d-a570-c0ffbfd4b804 => generated 1576 bytes in 181 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:29:59.450330 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-8d24a7df-a523-4fd8-a988-4c1cb0e3a20e tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:29:59.450929 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-8d24a7df-a523-4fd8-a988-4c1cb0e3a20e tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] [instance: 98e4cfeb-d49e-4888-a2c0-40a9e8af90d3] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:29:59.457407 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-8d24a7df-a523-4fd8-a988-4c1cb0e3a20e tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:29:59.457870 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-8d24a7df-a523-4fd8-a988-4c1cb0e3a20e tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:29:59.458304 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-8d24a7df-a523-4fd8-a988-4c1cb0e3a20e tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:29:59.459002 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-8d24a7df-a523-4fd8-a988-4c1cb0e3a20e tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:29:59.459458 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-8d24a7df-a523-4fd8-a988-4c1cb0e3a20e tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:29:59.459879 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-8d24a7df-a523-4fd8-a988-4c1cb0e3a20e tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:29:59.604059 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-8d24a7df-a523-4fd8-a988-4c1cb0e3a20e tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd945d512ae034ec08838d0a3afde2429', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '880ac4003248441c809a76cfd1b221eb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:29:59.612956 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-8d24a7df-a523-4fd8-a988-4c1cb0e3a20e tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] 10.222.0.52 "GET /compute/v2.1/servers/98e4cfeb-d49e-4888-a2c0-40a9e8af90d3" status: 200 len: 1735 microversion: 2.1 time: 0.166504 Aug 27 02:29:59.613594 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 376/759] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:29:59 2022] GET /compute/v2.1/servers/98e4cfeb-d49e-4888-a2c0-40a9e8af90d3 => generated 1735 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:29:59.924215 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-a8a71a53-1502-4b25-9e9e-0983d7d92091 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Calling method '>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:29:59.926593 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-a8a71a53-1502-4b25-9e9e-0983d7d92091 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] [instance: 2ad09ab6-ef1f-47a2-bf44-6ae046cc8645] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:29:59.937232 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-a8a71a53-1502-4b25-9e9e-0983d7d92091 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:29:59.937919 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-a8a71a53-1502-4b25-9e9e-0983d7d92091 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:29:59.938361 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-a8a71a53-1502-4b25-9e9e-0983d7d92091 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:29:59.939071 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-a8a71a53-1502-4b25-9e9e-0983d7d92091 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:29:59.939515 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-a8a71a53-1502-4b25-9e9e-0983d7d92091 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:29:59.939928 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-a8a71a53-1502-4b25-9e9e-0983d7d92091 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:29:59.975138 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-a8a71a53-1502-4b25-9e9e-0983d7d92091 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] 10.222.0.52 "GET /compute/v2.1/servers/2ad09ab6-ef1f-47a2-bf44-6ae046cc8645/os-volume_attachments" status: 200 len: 25 microversion: 2.1 time: 0.054541 Aug 27 02:29:59.976092 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 384/760] 10.222.0.52 () {58 vars in 1302 bytes} [Sat Aug 27 02:29:59 2022] GET /compute/v2.1/servers/2ad09ab6-ef1f-47a2-bf44-6ae046cc8645/os-volume_attachments => generated 25 bytes in 57 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Aug 27 02:30:00.051316 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-60c1c2b8-f66b-47ca-b5d8-444fb0736ce7 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:30:00.051909 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-60c1c2b8-f66b-47ca-b5d8-444fb0736ce7 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] [instance: 2ad09ab6-ef1f-47a2-bf44-6ae046cc8645] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:30:00.058246 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-60c1c2b8-f66b-47ca-b5d8-444fb0736ce7 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:30:00.058741 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-60c1c2b8-f66b-47ca-b5d8-444fb0736ce7 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:30:00.059167 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-60c1c2b8-f66b-47ca-b5d8-444fb0736ce7 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:30:00.059864 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-60c1c2b8-f66b-47ca-b5d8-444fb0736ce7 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:30:00.060309 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-60c1c2b8-f66b-47ca-b5d8-444fb0736ce7 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:30:00.060727 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-60c1c2b8-f66b-47ca-b5d8-444fb0736ce7 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:30:00.088910 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-60c1c2b8-f66b-47ca-b5d8-444fb0736ce7 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] [instance: 2ad09ab6-ef1f-47a2-bf44-6ae046cc8645] Going to try to terminate instance {{(pid=120863) delete /opt/stack/nova/nova/compute/api.py:2719}} Aug 27 02:30:00.168141 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-60c1c2b8-f66b-47ca-b5d8-444fb0736ce7 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] 10.222.0.52 "DELETE /compute/v2.1/servers/2ad09ab6-ef1f-47a2-bf44-6ae046cc8645" status: 204 len: 0 microversion: 2.1 time: 0.120585 Aug 27 02:30:00.168771 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 377/761] 10.222.0.52 () {58 vars in 1239 bytes} [Sat Aug 27 02:30:00 2022] DELETE /compute/v2.1/servers/2ad09ab6-ef1f-47a2-bf44-6ae046cc8645 => generated 0 bytes in 122 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Aug 27 02:30:00.180650 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-9e3c3a2a-2d26-4b00-b437-4c65858dae33 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Calling method '>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:30:00.181261 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-9e3c3a2a-2d26-4b00-b437-4c65858dae33 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] [instance: 2ad09ab6-ef1f-47a2-bf44-6ae046cc8645] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:30:00.188041 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-9e3c3a2a-2d26-4b00-b437-4c65858dae33 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:30:00.188615 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-9e3c3a2a-2d26-4b00-b437-4c65858dae33 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:30:00.189067 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-9e3c3a2a-2d26-4b00-b437-4c65858dae33 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:30:00.189792 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-9e3c3a2a-2d26-4b00-b437-4c65858dae33 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:30:00.190238 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-9e3c3a2a-2d26-4b00-b437-4c65858dae33 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:30:00.190652 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-9e3c3a2a-2d26-4b00-b437-4c65858dae33 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:30:00.266320 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-1b01cf5b-bdcd-447d-86e8-9dd03050f85d tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:30:00.267686 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-1b01cf5b-bdcd-447d-86e8-9dd03050f85d tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] [instance: 9941c49f-8626-461d-a570-c0ffbfd4b804] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:30:00.274924 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-1b01cf5b-bdcd-447d-86e8-9dd03050f85d tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:30:00.275587 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-1b01cf5b-bdcd-447d-86e8-9dd03050f85d tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:30:00.275935 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-1b01cf5b-bdcd-447d-86e8-9dd03050f85d tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:30:00.276658 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-1b01cf5b-bdcd-447d-86e8-9dd03050f85d tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:30:00.277170 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-1b01cf5b-bdcd-447d-86e8-9dd03050f85d tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:30:00.277584 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-1b01cf5b-bdcd-447d-86e8-9dd03050f85d tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:30:00.625327 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-9e3c3a2a-2d26-4b00-b437-4c65858dae33 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '26a86351583d49d090b3db09f127d49f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5d5ec8450c0941d18d63558baab54737', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:30:00.634255 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-9e3c3a2a-2d26-4b00-b437-4c65858dae33 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] 10.222.0.52 "GET /compute/v2.1/servers/2ad09ab6-ef1f-47a2-bf44-6ae046cc8645" status: 200 len: 1679 microversion: 2.1 time: 0.459017 Aug 27 02:30:00.634814 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 385/762] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:30:00 2022] GET /compute/v2.1/servers/2ad09ab6-ef1f-47a2-bf44-6ae046cc8645 => generated 1679 bytes in 460 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:30:00.651112 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-84fdf72c-0271-4ea1-9a52-c3f4bbdeef2c tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Calling method '>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:30:00.651803 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-84fdf72c-0271-4ea1-9a52-c3f4bbdeef2c tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] [instance: 98e4cfeb-d49e-4888-a2c0-40a9e8af90d3] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:30:00.653242 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-1b01cf5b-bdcd-447d-86e8-9dd03050f85d tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '642291c4daa04a6ab3fbce92a8be39c6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b2ddff853b7546788b645a0344c2297b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:30:00.664445 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-1b01cf5b-bdcd-447d-86e8-9dd03050f85d tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] 10.222.0.52 "GET /compute/v2.1/servers/9941c49f-8626-461d-a570-c0ffbfd4b804" status: 200 len: 1576 microversion: 2.1 time: 0.402089 Aug 27 02:30:00.665479 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-84fdf72c-0271-4ea1-9a52-c3f4bbdeef2c tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:30:00.665655 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 378/763] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:30:00 2022] GET /compute/v2.1/servers/9941c49f-8626-461d-a570-c0ffbfd4b804 => generated 1576 bytes in 404 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:30:00.665983 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-84fdf72c-0271-4ea1-9a52-c3f4bbdeef2c tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:30:00.666514 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-84fdf72c-0271-4ea1-9a52-c3f4bbdeef2c tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:30:00.667240 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-84fdf72c-0271-4ea1-9a52-c3f4bbdeef2c tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:30:00.667693 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-84fdf72c-0271-4ea1-9a52-c3f4bbdeef2c tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:30:00.668116 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-84fdf72c-0271-4ea1-9a52-c3f4bbdeef2c tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:30:00.807868 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-84fdf72c-0271-4ea1-9a52-c3f4bbdeef2c tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd945d512ae034ec08838d0a3afde2429', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '880ac4003248441c809a76cfd1b221eb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:30:00.816200 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-84fdf72c-0271-4ea1-9a52-c3f4bbdeef2c tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] 10.222.0.52 "GET /compute/v2.1/servers/98e4cfeb-d49e-4888-a2c0-40a9e8af90d3" status: 200 len: 1735 microversion: 2.1 time: 0.171218 Aug 27 02:30:00.816768 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 386/764] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:30:00 2022] GET /compute/v2.1/servers/98e4cfeb-d49e-4888-a2c0-40a9e8af90d3 => generated 1735 bytes in 181 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:30:01.655739 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-5dadf2bc-1016-4ab4-8c4c-7596b310cd4a tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:30:01.656349 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-5dadf2bc-1016-4ab4-8c4c-7596b310cd4a tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] [instance: 2ad09ab6-ef1f-47a2-bf44-6ae046cc8645] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:30:01.663323 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-5dadf2bc-1016-4ab4-8c4c-7596b310cd4a tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:30:01.663889 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-5dadf2bc-1016-4ab4-8c4c-7596b310cd4a tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:30:01.664419 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-5dadf2bc-1016-4ab4-8c4c-7596b310cd4a tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:30:01.665455 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-5dadf2bc-1016-4ab4-8c4c-7596b310cd4a tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:30:01.665906 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-5dadf2bc-1016-4ab4-8c4c-7596b310cd4a tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:30:01.666328 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-5dadf2bc-1016-4ab4-8c4c-7596b310cd4a tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:30:01.687312 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-e98be663-c25b-4b83-ae37-7425387be5af tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] Calling method '>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:30:01.687959 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-e98be663-c25b-4b83-ae37-7425387be5af tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] [instance: 9941c49f-8626-461d-a570-c0ffbfd4b804] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:30:01.694979 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-e98be663-c25b-4b83-ae37-7425387be5af tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:30:01.695712 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-e98be663-c25b-4b83-ae37-7425387be5af tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:30:01.696189 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-e98be663-c25b-4b83-ae37-7425387be5af tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:30:01.697000 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-e98be663-c25b-4b83-ae37-7425387be5af tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:30:01.697514 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-e98be663-c25b-4b83-ae37-7425387be5af tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:30:01.697993 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-e98be663-c25b-4b83-ae37-7425387be5af tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:30:01.876976 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-5dadf2bc-1016-4ab4-8c4c-7596b310cd4a tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '26a86351583d49d090b3db09f127d49f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5d5ec8450c0941d18d63558baab54737', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:30:01.885553 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-5dadf2bc-1016-4ab4-8c4c-7596b310cd4a tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] 10.222.0.52 "GET /compute/v2.1/servers/2ad09ab6-ef1f-47a2-bf44-6ae046cc8645" status: 200 len: 1679 microversion: 2.1 time: 0.233321 Aug 27 02:30:01.886449 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 379/765] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:30:01 2022] GET /compute/v2.1/servers/2ad09ab6-ef1f-47a2-bf44-6ae046cc8645 => generated 1679 bytes in 235 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:30:01.887802 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-e98be663-c25b-4b83-ae37-7425387be5af tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '642291c4daa04a6ab3fbce92a8be39c6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b2ddff853b7546788b645a0344c2297b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:30:01.890992 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-8665f07c-2fd7-41c4-8963-5fd779783862 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:30:01.891776 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-8665f07c-2fd7-41c4-8963-5fd779783862 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] [instance: 98e4cfeb-d49e-4888-a2c0-40a9e8af90d3] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:30:01.896362 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-e98be663-c25b-4b83-ae37-7425387be5af tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] 10.222.0.52 "GET /compute/v2.1/servers/9941c49f-8626-461d-a570-c0ffbfd4b804" status: 200 len: 1576 microversion: 2.1 time: 0.212948 Aug 27 02:30:01.897165 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 387/766] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:30:01 2022] GET /compute/v2.1/servers/9941c49f-8626-461d-a570-c0ffbfd4b804 => generated 1576 bytes in 215 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 27 02:30:01.897999 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-8665f07c-2fd7-41c4-8963-5fd779783862 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:30:01.898584 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-8665f07c-2fd7-41c4-8963-5fd779783862 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:30:01.899073 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-8665f07c-2fd7-41c4-8963-5fd779783862 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:30:01.899802 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-8665f07c-2fd7-41c4-8963-5fd779783862 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:30:01.900245 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-8665f07c-2fd7-41c4-8963-5fd779783862 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:30:01.900721 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-8665f07c-2fd7-41c4-8963-5fd779783862 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:30:02.048067 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-8665f07c-2fd7-41c4-8963-5fd779783862 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd945d512ae034ec08838d0a3afde2429', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '880ac4003248441c809a76cfd1b221eb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:30:02.059177 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-8665f07c-2fd7-41c4-8963-5fd779783862 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] 10.222.0.52 "GET /compute/v2.1/servers/98e4cfeb-d49e-4888-a2c0-40a9e8af90d3" status: 200 len: 1735 microversion: 2.1 time: 0.171601 Aug 27 02:30:02.059742 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 380/767] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:30:01 2022] GET /compute/v2.1/servers/98e4cfeb-d49e-4888-a2c0-40a9e8af90d3 => generated 1735 bytes in 173 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:30:02.907863 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-dd698b59-004a-4036-a26d-47aefd5bbba3 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Calling method '>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:30:02.908630 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-dd698b59-004a-4036-a26d-47aefd5bbba3 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] [instance: 2ad09ab6-ef1f-47a2-bf44-6ae046cc8645] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:30:02.917080 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-dd698b59-004a-4036-a26d-47aefd5bbba3 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:30:02.918091 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-dd698b59-004a-4036-a26d-47aefd5bbba3 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:30:02.918748 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-dd698b59-004a-4036-a26d-47aefd5bbba3 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:30:02.919346 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-42b6e15e-2a34-4a89-9d0e-c2ceb0d796bb tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:30:02.919955 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-dd698b59-004a-4036-a26d-47aefd5bbba3 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:30:02.920644 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-dd698b59-004a-4036-a26d-47aefd5bbba3 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:30:02.920791 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-42b6e15e-2a34-4a89-9d0e-c2ceb0d796bb tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] [instance: 9941c49f-8626-461d-a570-c0ffbfd4b804] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:30:02.921411 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-dd698b59-004a-4036-a26d-47aefd5bbba3 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:30:02.927852 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-42b6e15e-2a34-4a89-9d0e-c2ceb0d796bb tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:30:02.928587 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-42b6e15e-2a34-4a89-9d0e-c2ceb0d796bb tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:30:02.929011 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-42b6e15e-2a34-4a89-9d0e-c2ceb0d796bb tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:30:02.930299 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-42b6e15e-2a34-4a89-9d0e-c2ceb0d796bb tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:30:02.930922 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-42b6e15e-2a34-4a89-9d0e-c2ceb0d796bb tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:30:02.931498 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-42b6e15e-2a34-4a89-9d0e-c2ceb0d796bb tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:30:03.168442 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-42b6e15e-2a34-4a89-9d0e-c2ceb0d796bb tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '642291c4daa04a6ab3fbce92a8be39c6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b2ddff853b7546788b645a0344c2297b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:30:03.172263 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-dd698b59-004a-4036-a26d-47aefd5bbba3 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '26a86351583d49d090b3db09f127d49f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5d5ec8450c0941d18d63558baab54737', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:30:03.181738 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-42b6e15e-2a34-4a89-9d0e-c2ceb0d796bb tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] 10.222.0.52 "GET /compute/v2.1/servers/9941c49f-8626-461d-a570-c0ffbfd4b804" status: 200 len: 1576 microversion: 2.1 time: 0.266266 Aug 27 02:30:03.182615 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 381/768] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:30:02 2022] GET /compute/v2.1/servers/9941c49f-8626-461d-a570-c0ffbfd4b804 => generated 1576 bytes in 268 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:30:03.188256 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-dd698b59-004a-4036-a26d-47aefd5bbba3 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] 10.222.0.52 "GET /compute/v2.1/servers/2ad09ab6-ef1f-47a2-bf44-6ae046cc8645" status: 200 len: 1679 microversion: 2.1 time: 0.278994 Aug 27 02:30:03.189432 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 388/769] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:30:02 2022] GET /compute/v2.1/servers/2ad09ab6-ef1f-47a2-bf44-6ae046cc8645 => generated 1679 bytes in 286 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:30:03.200519 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-e3f0f758-3b1c-4d28-bb03-05a21e5921f3 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:30:03.201524 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-e3f0f758-3b1c-4d28-bb03-05a21e5921f3 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] [instance: 98e4cfeb-d49e-4888-a2c0-40a9e8af90d3] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:30:03.207992 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-e3f0f758-3b1c-4d28-bb03-05a21e5921f3 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:30:03.208479 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-e3f0f758-3b1c-4d28-bb03-05a21e5921f3 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:30:03.208910 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-e3f0f758-3b1c-4d28-bb03-05a21e5921f3 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:30:03.209642 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-e3f0f758-3b1c-4d28-bb03-05a21e5921f3 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:30:03.210089 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-e3f0f758-3b1c-4d28-bb03-05a21e5921f3 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:30:03.210512 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-e3f0f758-3b1c-4d28-bb03-05a21e5921f3 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:30:03.359748 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-e3f0f758-3b1c-4d28-bb03-05a21e5921f3 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd945d512ae034ec08838d0a3afde2429', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '880ac4003248441c809a76cfd1b221eb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:30:03.370095 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-e3f0f758-3b1c-4d28-bb03-05a21e5921f3 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] 10.222.0.52 "GET /compute/v2.1/servers/98e4cfeb-d49e-4888-a2c0-40a9e8af90d3" status: 200 len: 1735 microversion: 2.1 time: 0.186278 Aug 27 02:30:03.382236 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 382/770] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:30:03 2022] GET /compute/v2.1/servers/98e4cfeb-d49e-4888-a2c0-40a9e8af90d3 => generated 1735 bytes in 199 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:30:03.753724 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [req-7bad08ea-bf3f-4581-af70-c9627fe109e5 req-a64a6860-1247-456a-927b-dd4e3e8a54dc service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "2ad09ab6-ef1f-47a2-bf44-6ae046cc8645", "name": "network-vif-deleted", "tag": "e98d7081-9ec1-49da-9111-4786cc0b96ef"}]} {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 27 02:30:03.765758 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [req-7bad08ea-bf3f-4581-af70-c9627fe109e5 req-a64a6860-1247-456a-927b-dd4e3e8a54dc service nova] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:30:03.766544 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [req-7bad08ea-bf3f-4581-af70-c9627fe109e5 req-a64a6860-1247-456a-927b-dd4e3e8a54dc service nova] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:30:03.767245 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [req-7bad08ea-bf3f-4581-af70-c9627fe109e5 req-a64a6860-1247-456a-927b-dd4e3e8a54dc service nova] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:30:03.779705 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.compute.server_external_events [req-7bad08ea-bf3f-4581-af70-c9627fe109e5 req-a64a6860-1247-456a-927b-dd4e3e8a54dc service nova] Creating event network-vif-deleted:e98d7081-9ec1-49da-9111-4786cc0b96ef for instance 2ad09ab6-ef1f-47a2-bf44-6ae046cc8645 on hv-cinder-84630 Aug 27 02:30:03.785425 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [req-7bad08ea-bf3f-4581-af70-c9627fe109e5 req-a64a6860-1247-456a-927b-dd4e3e8a54dc service nova] 10.222.0.52 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.035388 Aug 27 02:30:03.786218 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 389/771] 10.222.0.52 () {64 vars in 1317 bytes} [Sat Aug 27 02:30:03 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 37 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 27 02:30:04.203713 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-6315d5b3-ce1b-41f8-a3eb-766c587427e8 tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:30:04.205352 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-6315d5b3-ce1b-41f8-a3eb-766c587427e8 tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] [instance: 9941c49f-8626-461d-a570-c0ffbfd4b804] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:30:04.212358 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-dee81181-3e4c-477e-8a6f-7a64e1d7879c tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Calling method '>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:30:04.213856 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-6315d5b3-ce1b-41f8-a3eb-766c587427e8 tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:30:04.214058 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-dee81181-3e4c-477e-8a6f-7a64e1d7879c tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] [instance: 2ad09ab6-ef1f-47a2-bf44-6ae046cc8645] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:30:04.214458 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-6315d5b3-ce1b-41f8-a3eb-766c587427e8 tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:30:04.214959 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-6315d5b3-ce1b-41f8-a3eb-766c587427e8 tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:30:04.215896 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-6315d5b3-ce1b-41f8-a3eb-766c587427e8 tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:30:04.217155 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-6315d5b3-ce1b-41f8-a3eb-766c587427e8 tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:30:04.217562 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-6315d5b3-ce1b-41f8-a3eb-766c587427e8 tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:30:04.220928 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-dee81181-3e4c-477e-8a6f-7a64e1d7879c tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:30:04.221434 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-dee81181-3e4c-477e-8a6f-7a64e1d7879c tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:30:04.222371 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-dee81181-3e4c-477e-8a6f-7a64e1d7879c tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:30:04.223044 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-dee81181-3e4c-477e-8a6f-7a64e1d7879c tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:30:04.223676 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-dee81181-3e4c-477e-8a6f-7a64e1d7879c tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:30:04.224083 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-dee81181-3e4c-477e-8a6f-7a64e1d7879c tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:30:04.317035 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-dee81181-3e4c-477e-8a6f-7a64e1d7879c tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '26a86351583d49d090b3db09f127d49f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5d5ec8450c0941d18d63558baab54737', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:30:04.326570 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-dee81181-3e4c-477e-8a6f-7a64e1d7879c tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] 10.222.0.52 "GET /compute/v2.1/servers/2ad09ab6-ef1f-47a2-bf44-6ae046cc8645" status: 200 len: 1612 microversion: 2.1 time: 0.120209 Aug 27 02:30:04.341161 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 390/772] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:30:04 2022] GET /compute/v2.1/servers/2ad09ab6-ef1f-47a2-bf44-6ae046cc8645 => generated 1612 bytes in 136 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:30:04.396037 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-40b4403a-d03a-4efb-9e61-2cfb3a469b08 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Calling method '>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:30:04.396681 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-40b4403a-d03a-4efb-9e61-2cfb3a469b08 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] [instance: 98e4cfeb-d49e-4888-a2c0-40a9e8af90d3] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:30:04.400056 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-6315d5b3-ce1b-41f8-a3eb-766c587427e8 tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '642291c4daa04a6ab3fbce92a8be39c6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b2ddff853b7546788b645a0344c2297b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:30:04.408082 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-40b4403a-d03a-4efb-9e61-2cfb3a469b08 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:30:04.408579 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-40b4403a-d03a-4efb-9e61-2cfb3a469b08 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:30:04.409011 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-40b4403a-d03a-4efb-9e61-2cfb3a469b08 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:30:04.409400 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-6315d5b3-ce1b-41f8-a3eb-766c587427e8 tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] 10.222.0.52 "GET /compute/v2.1/servers/9941c49f-8626-461d-a570-c0ffbfd4b804" status: 200 len: 1576 microversion: 2.1 time: 0.209338 Aug 27 02:30:04.413155 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 383/773] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:30:04 2022] GET /compute/v2.1/servers/9941c49f-8626-461d-a570-c0ffbfd4b804 => generated 1576 bytes in 211 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:30:04.413728 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-40b4403a-d03a-4efb-9e61-2cfb3a469b08 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:30:04.414231 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-40b4403a-d03a-4efb-9e61-2cfb3a469b08 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:30:04.415402 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-40b4403a-d03a-4efb-9e61-2cfb3a469b08 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:30:04.581635 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-40b4403a-d03a-4efb-9e61-2cfb3a469b08 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd945d512ae034ec08838d0a3afde2429', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '880ac4003248441c809a76cfd1b221eb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:30:04.591805 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-40b4403a-d03a-4efb-9e61-2cfb3a469b08 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] 10.222.0.52 "GET /compute/v2.1/servers/98e4cfeb-d49e-4888-a2c0-40a9e8af90d3" status: 200 len: 1715 microversion: 2.1 time: 0.200173 Aug 27 02:30:04.592566 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 391/774] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:30:04 2022] GET /compute/v2.1/servers/98e4cfeb-d49e-4888-a2c0-40a9e8af90d3 => generated 1715 bytes in 202 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:30:04.666799 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-edc125f1-7781-4204-a0a8-33b6e052c063 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:30:04.667342 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-edc125f1-7781-4204-a0a8-33b6e052c063 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] [instance: 98e4cfeb-d49e-4888-a2c0-40a9e8af90d3] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:30:04.674028 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-edc125f1-7781-4204-a0a8-33b6e052c063 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:30:04.674522 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-edc125f1-7781-4204-a0a8-33b6e052c063 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:30:04.674956 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-edc125f1-7781-4204-a0a8-33b6e052c063 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:30:04.675660 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-edc125f1-7781-4204-a0a8-33b6e052c063 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:30:04.676108 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-edc125f1-7781-4204-a0a8-33b6e052c063 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:30:04.676725 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-edc125f1-7781-4204-a0a8-33b6e052c063 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:30:04.783972 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.wsgi [None req-edc125f1-7781-4204-a0a8-33b6e052c063 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] HTTP exception thrown: Cannot 'detach_volume' instance 98e4cfeb-d49e-4888-a2c0-40a9e8af90d3 while it is in vm_state rescued Aug 27 02:30:04.786963 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-edc125f1-7781-4204-a0a8-33b6e052c063 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Request method failure captured: Aug 27 02:30:04.786963 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: request: DELETE /compute/v2.1/servers/98e4cfeb-d49e-4888-a2c0-40a9e8af90d3/os-volume_attachments/2a8726a0-0c8d-4001-aaca-d11c877d0d88 HTTP/1.1 Aug 27 02:30:04.786963 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: Accept: application/json Aug 27 02:30:04.786963 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: Accept-Encoding: identity Aug 27 02:30:04.786963 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: Connection: close Aug 27 02:30:04.786963 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: Content-Length: 0 Aug 27 02:30:04.786963 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: Content-Type: application/json Aug 27 02:30:04.786963 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: Host: 10.222.0.52 Aug 27 02:30:04.786963 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: Openstack-System-Scope: None Aug 27 02:30:04.786963 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: User-Agent: python-urllib3/1.26.11 Aug 27 02:30:04.786963 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-Auth-Token: gAAAAABjCX_mTgf5582eaPxg7VeVEFNV70u35gVl4tYbLlqoLLSOTdT51sTqNUwxXgJqsRZlTyjbNrsB5eN_l1XXJng0l2G5rQDKOgGEffO457OUUMgEUK1evfhiyjkwW8ZvB7sHsdxG4zKRGU2G452QU2-KddZbqwFS3zX2uIy2nVW08GLJmCc Aug 27 02:30:04.786963 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-Domain-Id: None Aug 27 02:30:04.786963 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-Domain-Name: None Aug 27 02:30:04.786963 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-Identity-Status: Confirmed Aug 27 02:30:04.786963 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-Is-Admin-Project: True Aug 27 02:30:04.786963 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-Project-Domain-Id: default Aug 27 02:30:04.786963 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-Project-Domain-Name: Default Aug 27 02:30:04.786963 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-Project-Id: 880ac4003248441c809a76cfd1b221eb Aug 27 02:30:04.786963 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-Project-Name: tempest-ServerRescueNegativeTestJSON-1551042004 Aug 27 02:30:04.786963 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-Role: reader,member Aug 27 02:30:04.786963 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-Roles: reader,member Aug 27 02:30:04.788391 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-Service-Catalog: [{"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.52:8080", "publicURL": "http://10.222.0.52:8080/v1/AUTH_880ac4003248441c809a76cfd1b221eb"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52:9696/networking"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/compute/v2/880ac4003248441c809a76cfd1b221eb"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/image"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/volume/v3/880ac4003248441c809a76cfd1b221eb"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/identity"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/volume/v3/880ac4003248441c809a76cfd1b221eb"}]}] Aug 27 02:30:04.788391 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-Tenant: tempest-ServerRescueNegativeTestJSON-1551042004 Aug 27 02:30:04.788391 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-Tenant-Id: 880ac4003248441c809a76cfd1b221eb Aug 27 02:30:04.788391 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-Tenant-Name: tempest-ServerRescueNegativeTestJSON-1551042004 Aug 27 02:30:04.788391 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-User: tempest-ServerRescueNegativeTestJSON-1551042004-project Aug 27 02:30:04.788391 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-User-Domain-Id: default Aug 27 02:30:04.788391 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-User-Domain-Name: Default Aug 27 02:30:04.788391 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-User-Id: d945d512ae034ec08838d0a3afde2429 Aug 27 02:30:04.788391 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-User-Name: tempest-ServerRescueNegativeTestJSON-1551042004-project Aug 27 02:30:04.788391 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: method: > Aug 27 02:30:04.788391 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: exception: Cannot 'detach_volume' instance 98e4cfeb-d49e-4888-a2c0-40a9e8af90d3 while it is in vm_state rescued Aug 27 02:30:04.789042 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: action_args: {'server_id': '98e4cfeb-d49e-4888-a2c0-40a9e8af90d3', 'id': '2a8726a0-0c8d-4001-aaca-d11c877d0d88'} Aug 27 02:30:04.789042 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Aug 27 02:30:04.789042 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 27 02:30:04.789042 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 565, in delete Aug 27 02:30:04.789042 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi self.compute_api.detach_volume(context, instance, volume) Aug 27 02:30:04.789042 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 241, in inner Aug 27 02:30:04.789042 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Aug 27 02:30:04.789042 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 148, in inner Aug 27 02:30:04.789042 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi raise exception.InstanceInvalidState( Aug 27 02:30:04.789042 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi nova.exception.InstanceInvalidState: Instance 98e4cfeb-d49e-4888-a2c0-40a9e8af90d3 in vm_state rescued. Cannot detach_volume while the instance is in this state. Aug 27 02:30:04.789042 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi Aug 27 02:30:04.789042 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 27 02:30:04.789042 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi Aug 27 02:30:04.789042 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 27 02:30:04.789042 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 27 02:30:04.789042 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 27 02:30:04.789042 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Aug 27 02:30:04.789042 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Aug 27 02:30:04.789042 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Aug 27 02:30:04.789042 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Aug 27 02:30:04.792567 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 573, in delete Aug 27 02:30:04.792567 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi common.raise_http_conflict_for_instance_invalid_state(state_error, Aug 27 02:30:04.792567 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 347, in raise_http_conflict_for_instance_invalid_state Aug 27 02:30:04.792567 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPConflict(explanation=msg) Aug 27 02:30:04.792567 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi webob.exc.HTTPConflict: Cannot 'detach_volume' instance 98e4cfeb-d49e-4888-a2c0-40a9e8af90d3 while it is in vm_state rescued Aug 27 02:30:04.792567 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi Aug 27 02:30:04.792567 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 27 02:30:04.792567 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi Aug 27 02:30:04.792567 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 27 02:30:04.792567 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 27 02:30:04.792567 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 27 02:30:04.792567 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Aug 27 02:30:04.792567 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Aug 27 02:30:04.792567 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Cannot 'detach_volume' instance 98e4cfeb-d49e-4888-a2c0-40a9e8af90d3 while it is in vm_state rescued Aug 27 02:30:04.792567 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi Aug 27 02:30:04.792567 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-edc125f1-7781-4204-a0a8-33b6e052c063 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Returning 409 to user: Cannot 'detach_volume' instance 98e4cfeb-d49e-4888-a2c0-40a9e8af90d3 while it is in vm_state rescued {{(pid=120863) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Aug 27 02:30:04.793632 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-edc125f1-7781-4204-a0a8-33b6e052c063 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] 10.222.0.52 "DELETE /compute/v2.1/servers/98e4cfeb-d49e-4888-a2c0-40a9e8af90d3/os-volume_attachments/2a8726a0-0c8d-4001-aaca-d11c877d0d88" status: 409 len: 152 microversion: 2.1 time: 0.126986 Aug 27 02:30:04.793632 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 384/775] 10.222.0.52 () {58 vars in 1416 bytes} [Sat Aug 27 02:30:04 2022] DELETE /compute/v2.1/servers/98e4cfeb-d49e-4888-a2c0-40a9e8af90d3/os-volume_attachments/2a8726a0-0c8d-4001-aaca-d11c877d0d88 => generated 152 bytes in 134 msecs (HTTP/1.1 409) 9 headers in 378 bytes (1 switches on core 0) Aug 27 02:30:04.801001 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-16254dc6-fe1e-4872-8a31-92e1884b832d tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Action: 'action', calling method: >, body: {"unrescue": {}} {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 27 02:30:04.801001 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-16254dc6-fe1e-4872-8a31-92e1884b832d tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] [instance: 98e4cfeb-d49e-4888-a2c0-40a9e8af90d3] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:30:04.807325 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-16254dc6-fe1e-4872-8a31-92e1884b832d tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:30:04.807814 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-16254dc6-fe1e-4872-8a31-92e1884b832d tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:30:04.808292 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-16254dc6-fe1e-4872-8a31-92e1884b832d tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:30:04.808945 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-16254dc6-fe1e-4872-8a31-92e1884b832d tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:30:04.809449 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-16254dc6-fe1e-4872-8a31-92e1884b832d tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:30:04.809867 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-16254dc6-fe1e-4872-8a31-92e1884b832d tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:30:04.850731 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG dbcounter [-] [120864] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=120864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 27 02:30:04.882151 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-16254dc6-fe1e-4872-8a31-92e1884b832d tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] 10.222.0.52 "POST /compute/v2.1/servers/98e4cfeb-d49e-4888-a2c0-40a9e8af90d3/action" status: 202 len: 0 microversion: 2.1 time: 0.085837 Aug 27 02:30:04.882784 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 392/776] 10.222.0.52 () {60 vars in 1278 bytes} [Sat Aug 27 02:30:04 2022] POST /compute/v2.1/servers/98e4cfeb-d49e-4888-a2c0-40a9e8af90d3/action => generated 0 bytes in 88 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Aug 27 02:30:04.892386 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-b08c8349-cf6e-4052-9da0-e3d8ae2cdb24 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:30:04.893938 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-b08c8349-cf6e-4052-9da0-e3d8ae2cdb24 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] [instance: 98e4cfeb-d49e-4888-a2c0-40a9e8af90d3] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:30:04.901435 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-b08c8349-cf6e-4052-9da0-e3d8ae2cdb24 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:30:04.902013 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-b08c8349-cf6e-4052-9da0-e3d8ae2cdb24 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:30:04.902525 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-b08c8349-cf6e-4052-9da0-e3d8ae2cdb24 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:30:04.903304 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-b08c8349-cf6e-4052-9da0-e3d8ae2cdb24 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:30:04.903846 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-b08c8349-cf6e-4052-9da0-e3d8ae2cdb24 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:30:04.904338 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-b08c8349-cf6e-4052-9da0-e3d8ae2cdb24 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:30:05.051167 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-b08c8349-cf6e-4052-9da0-e3d8ae2cdb24 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd945d512ae034ec08838d0a3afde2429', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '880ac4003248441c809a76cfd1b221eb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:30:05.062441 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-b08c8349-cf6e-4052-9da0-e3d8ae2cdb24 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] 10.222.0.52 "GET /compute/v2.1/servers/98e4cfeb-d49e-4888-a2c0-40a9e8af90d3" status: 200 len: 1723 microversion: 2.1 time: 0.174576 Aug 27 02:30:05.062992 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 385/777] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:30:04 2022] GET /compute/v2.1/servers/98e4cfeb-d49e-4888-a2c0-40a9e8af90d3 => generated 1723 bytes in 176 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:30:05.350375 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-5efaf16c-ea53-4276-b5e2-bb5505e8e4d0 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Calling method '>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:30:05.351409 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-5efaf16c-ea53-4276-b5e2-bb5505e8e4d0 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] [instance: 2ad09ab6-ef1f-47a2-bf44-6ae046cc8645] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:30:05.358015 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-5efaf16c-ea53-4276-b5e2-bb5505e8e4d0 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:30:05.358547 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-5efaf16c-ea53-4276-b5e2-bb5505e8e4d0 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:30:05.358945 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-5efaf16c-ea53-4276-b5e2-bb5505e8e4d0 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:30:05.359679 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-5efaf16c-ea53-4276-b5e2-bb5505e8e4d0 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:30:05.360132 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-5efaf16c-ea53-4276-b5e2-bb5505e8e4d0 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:30:05.360531 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-5efaf16c-ea53-4276-b5e2-bb5505e8e4d0 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:30:05.393069 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-5efaf16c-ea53-4276-b5e2-bb5505e8e4d0 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:30:05.393588 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-5efaf16c-ea53-4276-b5e2-bb5505e8e4d0 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:30:05.394019 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-5efaf16c-ea53-4276-b5e2-bb5505e8e4d0 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:30:05.395217 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.objects.instance [None req-5efaf16c-ea53-4276-b5e2-bb5505e8e4d0 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lazy-loading 'fault' on Instance uuid 2ad09ab6-ef1f-47a2-bf44-6ae046cc8645 {{(pid=120864) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Aug 27 02:30:05.433030 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-d90882ac-0e9d-4feb-83f4-295556e2efbb tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:30:05.433667 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-d90882ac-0e9d-4feb-83f4-295556e2efbb tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] [instance: 9941c49f-8626-461d-a570-c0ffbfd4b804] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:30:05.440374 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-d90882ac-0e9d-4feb-83f4-295556e2efbb tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:30:05.440844 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-d90882ac-0e9d-4feb-83f4-295556e2efbb tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:30:05.441331 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-d90882ac-0e9d-4feb-83f4-295556e2efbb tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:30:05.442072 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-d90882ac-0e9d-4feb-83f4-295556e2efbb tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:30:05.442519 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-d90882ac-0e9d-4feb-83f4-295556e2efbb tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:30:05.442931 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-d90882ac-0e9d-4feb-83f4-295556e2efbb tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:30:05.447086 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-5efaf16c-ea53-4276-b5e2-bb5505e8e4d0 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '26a86351583d49d090b3db09f127d49f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5d5ec8450c0941d18d63558baab54737', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:30:05.458882 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-5efaf16c-ea53-4276-b5e2-bb5505e8e4d0 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] 10.222.0.52 "GET /compute/v2.1/servers/2ad09ab6-ef1f-47a2-bf44-6ae046cc8645" status: 200 len: 1331 microversion: 2.1 time: 0.113593 Aug 27 02:30:05.459464 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 393/778] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:30:05 2022] GET /compute/v2.1/servers/2ad09ab6-ef1f-47a2-bf44-6ae046cc8645 => generated 1331 bytes in 115 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:30:05.597747 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-d90882ac-0e9d-4feb-83f4-295556e2efbb tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '642291c4daa04a6ab3fbce92a8be39c6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b2ddff853b7546788b645a0344c2297b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:30:05.609131 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-d90882ac-0e9d-4feb-83f4-295556e2efbb tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] 10.222.0.52 "GET /compute/v2.1/servers/9941c49f-8626-461d-a570-c0ffbfd4b804" status: 200 len: 1576 microversion: 2.1 time: 0.180283 Aug 27 02:30:05.609708 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 386/779] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:30:05 2022] GET /compute/v2.1/servers/9941c49f-8626-461d-a570-c0ffbfd4b804 => generated 1576 bytes in 182 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:30:06.086846 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-7d6dac2a-49f4-4b61-9b23-0330407cd075 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Calling method '>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:30:06.087465 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-7d6dac2a-49f4-4b61-9b23-0330407cd075 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] [instance: 98e4cfeb-d49e-4888-a2c0-40a9e8af90d3] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:30:06.094450 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-7d6dac2a-49f4-4b61-9b23-0330407cd075 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:30:06.095044 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-7d6dac2a-49f4-4b61-9b23-0330407cd075 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:30:06.095477 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-7d6dac2a-49f4-4b61-9b23-0330407cd075 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:30:06.096367 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-7d6dac2a-49f4-4b61-9b23-0330407cd075 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:30:06.096959 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-7d6dac2a-49f4-4b61-9b23-0330407cd075 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:30:06.097505 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-7d6dac2a-49f4-4b61-9b23-0330407cd075 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:30:06.247766 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-7d6dac2a-49f4-4b61-9b23-0330407cd075 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd945d512ae034ec08838d0a3afde2429', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '880ac4003248441c809a76cfd1b221eb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:30:06.262307 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-7d6dac2a-49f4-4b61-9b23-0330407cd075 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] 10.222.0.52 "GET /compute/v2.1/servers/98e4cfeb-d49e-4888-a2c0-40a9e8af90d3" status: 200 len: 1723 microversion: 2.1 time: 0.180086 Aug 27 02:30:06.263813 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 394/780] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:30:06 2022] GET /compute/v2.1/servers/98e4cfeb-d49e-4888-a2c0-40a9e8af90d3 => generated 1723 bytes in 182 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:30:06.483039 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-9f56669b-4c99-4403-8edb-a4cdbae5c6c4 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:30:06.484683 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-9f56669b-4c99-4403-8edb-a4cdbae5c6c4 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] [instance: 2ad09ab6-ef1f-47a2-bf44-6ae046cc8645] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:30:06.492396 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-9f56669b-4c99-4403-8edb-a4cdbae5c6c4 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:30:06.493697 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-9f56669b-4c99-4403-8edb-a4cdbae5c6c4 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:30:06.494273 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-9f56669b-4c99-4403-8edb-a4cdbae5c6c4 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:30:06.494880 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-9f56669b-4c99-4403-8edb-a4cdbae5c6c4 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:30:06.495382 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-9f56669b-4c99-4403-8edb-a4cdbae5c6c4 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:30:06.495809 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-9f56669b-4c99-4403-8edb-a4cdbae5c6c4 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:30:06.501165 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG dbcounter [-] [120863] Writing DB stats nova_cell1:SELECT=97,nova_cell1:INSERT=2,nova_cell1:UPDATE=1 {{(pid=120863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 27 02:30:06.512542 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.wsgi [None req-9f56669b-4c99-4403-8edb-a4cdbae5c6c4 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] HTTP exception thrown: Instance 2ad09ab6-ef1f-47a2-bf44-6ae046cc8645 could not be found. Aug 27 02:30:06.514342 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-9f56669b-4c99-4403-8edb-a4cdbae5c6c4 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Request method failure captured: Aug 27 02:30:06.514342 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: request: GET /compute/v2.1/servers/2ad09ab6-ef1f-47a2-bf44-6ae046cc8645 HTTP/1.1 Aug 27 02:30:06.514342 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: Accept: application/json Aug 27 02:30:06.514342 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: Accept-Encoding: identity Aug 27 02:30:06.514342 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: Connection: close Aug 27 02:30:06.514342 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: Content-Length: 0 Aug 27 02:30:06.514342 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: Content-Type: application/json Aug 27 02:30:06.514342 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: Host: 10.222.0.52 Aug 27 02:30:06.514342 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: Openstack-System-Scope: None Aug 27 02:30:06.514342 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: User-Agent: python-urllib3/1.26.11 Aug 27 02:30:06.514342 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-Auth-Token: gAAAAABjCX_pkOb4-Fffq6Hbe0cfjlj1ARL1JmORSzrYWucWmWgZ67LEorDD3IWpPkj3UP534US8GW_nja0iAKwcdxSx3e5teQzZMdrCvOvA5aSGJO8iodOjhfOIsl8L0d8om8kj8_3q-8uTHxKupSTnIQy_J-f1kC4e22VNejIIiExDsGitats Aug 27 02:30:06.514342 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-Domain-Id: None Aug 27 02:30:06.514342 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-Domain-Name: None Aug 27 02:30:06.514342 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-Identity-Status: Confirmed Aug 27 02:30:06.514342 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-Is-Admin-Project: True Aug 27 02:30:06.514342 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-Project-Domain-Id: default Aug 27 02:30:06.514342 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-Project-Domain-Name: Default Aug 27 02:30:06.514342 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-Project-Id: 5d5ec8450c0941d18d63558baab54737 Aug 27 02:30:06.514342 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-Project-Name: tempest-AttachVolumeNegativeTest-1988669364 Aug 27 02:30:06.514342 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-Role: reader,member Aug 27 02:30:06.514342 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-Roles: reader,member Aug 27 02:30:06.516335 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-Service-Catalog: [{"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.52:8080", "publicURL": "http://10.222.0.52:8080/v1/AUTH_5d5ec8450c0941d18d63558baab54737"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52:9696/networking"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/compute/v2/5d5ec8450c0941d18d63558baab54737"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/image"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/volume/v3/5d5ec8450c0941d18d63558baab54737"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/identity"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/volume/v3/5d5ec8450c0941d18d63558baab54737"}]}] Aug 27 02:30:06.516335 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-Tenant: tempest-AttachVolumeNegativeTest-1988669364 Aug 27 02:30:06.516335 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-Tenant-Id: 5d5ec8450c0941d18d63558baab54737 Aug 27 02:30:06.516335 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-1988669364 Aug 27 02:30:06.516335 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-User: tempest-AttachVolumeNegativeTest-1988669364-project Aug 27 02:30:06.516335 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-User-Domain-Id: default Aug 27 02:30:06.516335 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-User-Domain-Name: Default Aug 27 02:30:06.516335 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-User-Id: 26a86351583d49d090b3db09f127d49f Aug 27 02:30:06.516335 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-User-Name: tempest-AttachVolumeNegativeTest-1988669364-project Aug 27 02:30:06.516335 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: method: > Aug 27 02:30:06.516335 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: exception: Instance 2ad09ab6-ef1f-47a2-bf44-6ae046cc8645 could not be found. Aug 27 02:30:06.516335 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: action_args: {'id': '2ad09ab6-ef1f-47a2-bf44-6ae046cc8645'} Aug 27 02:30:06.517499 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Aug 27 02:30:06.517499 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 27 02:30:06.517499 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Aug 27 02:30:06.517499 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Aug 27 02:30:06.517499 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Aug 27 02:30:06.517499 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Aug 27 02:30:06.517499 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Aug 27 02:30:06.517499 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Aug 27 02:30:06.517499 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Aug 27 02:30:06.517499 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Aug 27 02:30:06.517499 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 2ad09ab6-ef1f-47a2-bf44-6ae046cc8645 could not be found. Aug 27 02:30:06.517499 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi Aug 27 02:30:06.517499 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 27 02:30:06.517499 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi Aug 27 02:30:06.517499 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 27 02:30:06.517499 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 27 02:30:06.517499 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 27 02:30:06.517499 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Aug 27 02:30:06.517499 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Aug 27 02:30:06.517499 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Aug 27 02:30:06.519268 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Aug 27 02:30:06.519268 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Aug 27 02:30:06.519268 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi instance = self._get_server( Aug 27 02:30:06.519268 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Aug 27 02:30:06.519268 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Aug 27 02:30:06.519268 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Aug 27 02:30:06.519268 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Aug 27 02:30:06.519268 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 2ad09ab6-ef1f-47a2-bf44-6ae046cc8645 could not be found. Aug 27 02:30:06.519268 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi Aug 27 02:30:06.519268 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 27 02:30:06.519268 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi Aug 27 02:30:06.519268 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 27 02:30:06.519268 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 27 02:30:06.519268 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 27 02:30:06.519268 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Aug 27 02:30:06.519268 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Aug 27 02:30:06.519268 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 2ad09ab6-ef1f-47a2-bf44-6ae046cc8645 could not be found. Aug 27 02:30:06.519268 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi Aug 27 02:30:06.520970 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-9f56669b-4c99-4403-8edb-a4cdbae5c6c4 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Returning 404 to user: Instance 2ad09ab6-ef1f-47a2-bf44-6ae046cc8645 could not be found. {{(pid=120863) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Aug 27 02:30:06.520970 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-9f56669b-4c99-4403-8edb-a4cdbae5c6c4 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] 10.222.0.52 "GET /compute/v2.1/servers/2ad09ab6-ef1f-47a2-bf44-6ae046cc8645" status: 404 len: 111 microversion: 2.1 time: 0.038717 Aug 27 02:30:06.520970 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 387/781] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:30:06 2022] GET /compute/v2.1/servers/2ad09ab6-ef1f-47a2-bf44-6ae046cc8645 => generated 111 bytes in 41 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Aug 27 02:30:06.526311 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-24228d05-71cf-4f0c-9c1e-d45538ee4d3a tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Calling method '.version_select at 0x7f6b1e8a8ee0>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:30:06.555667 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-24228d05-71cf-4f0c-9c1e-d45538ee4d3a tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] 10.222.0.52 "DELETE /compute/v2.1/os-keypairs/tempest-keypair-1257325292" status: 202 len: 0 microversion: 2.1 time: 0.033170 Aug 27 02:30:06.556256 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 395/782] 10.222.0.52 () {58 vars in 1221 bytes} [Sat Aug 27 02:30:06 2022] DELETE /compute/v2.1/os-keypairs/tempest-keypair-1257325292 => generated 0 bytes in 34 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Aug 27 02:30:06.636881 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-81cc34e5-fa1b-4a38-81a6-ad1cb2cc52bf tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:30:06.637604 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-81cc34e5-fa1b-4a38-81a6-ad1cb2cc52bf tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] [instance: 9941c49f-8626-461d-a570-c0ffbfd4b804] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:30:06.645358 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-81cc34e5-fa1b-4a38-81a6-ad1cb2cc52bf tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:30:06.646010 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-81cc34e5-fa1b-4a38-81a6-ad1cb2cc52bf tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:30:06.647397 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-81cc34e5-fa1b-4a38-81a6-ad1cb2cc52bf tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:30:06.648284 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-81cc34e5-fa1b-4a38-81a6-ad1cb2cc52bf tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:30:06.649200 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-81cc34e5-fa1b-4a38-81a6-ad1cb2cc52bf tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:30:06.650554 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-81cc34e5-fa1b-4a38-81a6-ad1cb2cc52bf tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:30:06.843966 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-81cc34e5-fa1b-4a38-81a6-ad1cb2cc52bf tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '642291c4daa04a6ab3fbce92a8be39c6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b2ddff853b7546788b645a0344c2297b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:30:06.852101 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-81cc34e5-fa1b-4a38-81a6-ad1cb2cc52bf tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] 10.222.0.52 "GET /compute/v2.1/servers/9941c49f-8626-461d-a570-c0ffbfd4b804" status: 200 len: 1576 microversion: 2.1 time: 0.221843 Aug 27 02:30:06.853043 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 388/783] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:30:06 2022] GET /compute/v2.1/servers/9941c49f-8626-461d-a570-c0ffbfd4b804 => generated 1576 bytes in 225 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:30:07.284926 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-a495565d-af3e-437a-9898-cb3d31db61b2 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Calling method '>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:30:07.285854 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-a495565d-af3e-437a-9898-cb3d31db61b2 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] [instance: 98e4cfeb-d49e-4888-a2c0-40a9e8af90d3] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:30:07.292030 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-a495565d-af3e-437a-9898-cb3d31db61b2 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:30:07.292518 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-a495565d-af3e-437a-9898-cb3d31db61b2 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:30:07.292948 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-a495565d-af3e-437a-9898-cb3d31db61b2 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:30:07.293668 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-a495565d-af3e-437a-9898-cb3d31db61b2 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:30:07.294118 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-a495565d-af3e-437a-9898-cb3d31db61b2 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:30:07.294535 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-a495565d-af3e-437a-9898-cb3d31db61b2 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:30:07.349261 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-28622e13-a6b1-4389-b858-9b07ed58155e tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Action: 'create', calling method: .version_select at 0x7f6b1b052af0>, body: {"keypair": {"name": "tempest-keypair-527069421", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBL09zdm3VadV27+Vl5zjn9xkyQTuVqg9ymMlsUk3WNJ9fyC/hAAPBfFTNEHZx9/tfGrdyGjOnJWDLWMkZfyT6Xuyb7QVVL0f8SZeldx4iNc/PUgsz3ckkvZ1ltIbclK39w=="}} {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 27 02:30:07.368247 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.quota [None req-28622e13-a6b1-4389-b858-9b07ed58155e tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Getting quotas for project 5d5ec8450c0941d18d63558baab54737. Resources: {'key_pairs'} {{(pid=120863) _get_quotas /opt/stack/nova/nova/quota.py:393}} Aug 27 02:30:07.373087 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.quota [None req-28622e13-a6b1-4389-b858-9b07ed58155e tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Getting quotas for user 26a86351583d49d090b3db09f127d49f and project 5d5ec8450c0941d18d63558baab54737. Resources: {'key_pairs'} {{(pid=120863) _get_quotas /opt/stack/nova/nova/quota.py:383}} Aug 27 02:30:07.401304 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-28622e13-a6b1-4389-b858-9b07ed58155e tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] 10.222.0.52 "POST /compute/v2.1/os-keypairs" status: 200 len: 385 microversion: 2.1 time: 0.055933 Aug 27 02:30:07.401888 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 389/784] 10.222.0.52 () {60 vars in 1159 bytes} [Sat Aug 27 02:30:07 2022] POST /compute/v2.1/os-keypairs => generated 385 bytes in 57 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 27 02:30:07.443216 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-a495565d-af3e-437a-9898-cb3d31db61b2 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd945d512ae034ec08838d0a3afde2429', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '880ac4003248441c809a76cfd1b221eb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:30:07.452731 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-a495565d-af3e-437a-9898-cb3d31db61b2 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] 10.222.0.52 "GET /compute/v2.1/servers/98e4cfeb-d49e-4888-a2c0-40a9e8af90d3" status: 200 len: 1723 microversion: 2.1 time: 0.171753 Aug 27 02:30:07.453350 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 396/785] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:30:07 2022] GET /compute/v2.1/servers/98e4cfeb-d49e-4888-a2c0-40a9e8af90d3 => generated 1723 bytes in 173 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:30:07.876043 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-31f7e8b4-ca5c-4e59-af1e-bec13d2dc97a tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:30:07.876762 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-31f7e8b4-ca5c-4e59-af1e-bec13d2dc97a tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] [instance: 9941c49f-8626-461d-a570-c0ffbfd4b804] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:30:07.884582 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-31f7e8b4-ca5c-4e59-af1e-bec13d2dc97a tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:30:07.885429 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-31f7e8b4-ca5c-4e59-af1e-bec13d2dc97a tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:30:07.885898 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-31f7e8b4-ca5c-4e59-af1e-bec13d2dc97a tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:30:07.886863 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-31f7e8b4-ca5c-4e59-af1e-bec13d2dc97a tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:30:07.887380 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-31f7e8b4-ca5c-4e59-af1e-bec13d2dc97a tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:30:07.887896 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-31f7e8b4-ca5c-4e59-af1e-bec13d2dc97a tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:30:08.108559 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-31f7e8b4-ca5c-4e59-af1e-bec13d2dc97a tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '642291c4daa04a6ab3fbce92a8be39c6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b2ddff853b7546788b645a0344c2297b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:30:08.117367 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-31f7e8b4-ca5c-4e59-af1e-bec13d2dc97a tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] 10.222.0.52 "GET /compute/v2.1/servers/9941c49f-8626-461d-a570-c0ffbfd4b804" status: 200 len: 1576 microversion: 2.1 time: 0.245990 Aug 27 02:30:08.134546 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 390/786] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:30:07 2022] GET /compute/v2.1/servers/9941c49f-8626-461d-a570-c0ffbfd4b804 => generated 1576 bytes in 264 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:30:08.184007 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [req-15a51025-4069-4f1c-8757-2163209ea9f2 req-db3d8af3-eb3e-4e04-8d33-e4b35583aed2 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "9941c49f-8626-461d-a570-c0ffbfd4b804", "name": "network-vif-plugged", "status": "completed", "tag": "89f4d982-1807-4e80-b9ef-c7bc0025f1b0"}]} {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 27 02:30:08.196667 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [req-15a51025-4069-4f1c-8757-2163209ea9f2 req-db3d8af3-eb3e-4e04-8d33-e4b35583aed2 service nova] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:30:08.197231 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [req-15a51025-4069-4f1c-8757-2163209ea9f2 req-db3d8af3-eb3e-4e04-8d33-e4b35583aed2 service nova] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:30:08.197940 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [req-15a51025-4069-4f1c-8757-2163209ea9f2 req-db3d8af3-eb3e-4e04-8d33-e4b35583aed2 service nova] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:30:08.212550 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.compute.server_external_events [req-15a51025-4069-4f1c-8757-2163209ea9f2 req-db3d8af3-eb3e-4e04-8d33-e4b35583aed2 service nova] Creating event network-vif-plugged:89f4d982-1807-4e80-b9ef-c7bc0025f1b0 for instance 9941c49f-8626-461d-a570-c0ffbfd4b804 on hv-cinder-84630 Aug 27 02:30:08.217687 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [req-15a51025-4069-4f1c-8757-2163209ea9f2 req-db3d8af3-eb3e-4e04-8d33-e4b35583aed2 service nova] 10.222.0.52 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.037822 Aug 27 02:30:08.218277 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 397/787] 10.222.0.52 () {64 vars in 1317 bytes} [Sat Aug 27 02:30:08 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 39 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Aug 27 02:30:08.475470 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-f4d360ed-80af-4be0-a948-09ad05547597 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:30:08.476326 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-f4d360ed-80af-4be0-a948-09ad05547597 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] [instance: 98e4cfeb-d49e-4888-a2c0-40a9e8af90d3] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:30:08.482914 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-f4d360ed-80af-4be0-a948-09ad05547597 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:30:08.483393 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-f4d360ed-80af-4be0-a948-09ad05547597 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:30:08.483821 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-f4d360ed-80af-4be0-a948-09ad05547597 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:30:08.484517 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-f4d360ed-80af-4be0-a948-09ad05547597 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:30:08.484964 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-f4d360ed-80af-4be0-a948-09ad05547597 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:30:08.485405 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-f4d360ed-80af-4be0-a948-09ad05547597 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:30:08.710716 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-f4d360ed-80af-4be0-a948-09ad05547597 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd945d512ae034ec08838d0a3afde2429', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '880ac4003248441c809a76cfd1b221eb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:30:08.722762 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-f4d360ed-80af-4be0-a948-09ad05547597 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] 10.222.0.52 "GET /compute/v2.1/servers/98e4cfeb-d49e-4888-a2c0-40a9e8af90d3" status: 200 len: 1723 microversion: 2.1 time: 0.251487 Aug 27 02:30:08.723338 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 391/788] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:30:08 2022] GET /compute/v2.1/servers/98e4cfeb-d49e-4888-a2c0-40a9e8af90d3 => generated 1723 bytes in 253 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:30:08.926812 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-af499363-fa18-4de9-9511-8f6400655ba5 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-AttachVolumeNegativeTest-server-502933729", "imageRef": "9fc30b36-b827-46f9-892d-30e808bc2b8a", "flavorRef": "452", "adminPass": "***", "networks": [{"uuid": "134184f4-d632-4173-b77c-1566bf4eb0fb"}], "security_groups": [{"name": "tempest-securitygroup--1976556435"}], "key_name": "tempest-keypair-527069421", "user_data": "}} {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 27 02:30:09.037435 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.network.neutron [None req-af499363-fa18-4de9-9511-8f6400655ba5 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] validate_networks() for [('134184f4-d632-4173-b77c-1566bf4eb0fb', None, None, None, None, None)] {{(pid=120864) validate_networks /opt/stack/nova/nova/network/neutron.py:2626}} Aug 27 02:30:09.140680 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-87bcd5e2-52a0-4d37-a298-63765bdfcd06 tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:30:09.141415 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-87bcd5e2-52a0-4d37-a298-63765bdfcd06 tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] [instance: 9941c49f-8626-461d-a570-c0ffbfd4b804] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:30:09.148100 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-87bcd5e2-52a0-4d37-a298-63765bdfcd06 tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:30:09.149026 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-87bcd5e2-52a0-4d37-a298-63765bdfcd06 tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:30:09.149499 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-87bcd5e2-52a0-4d37-a298-63765bdfcd06 tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:30:09.150238 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-87bcd5e2-52a0-4d37-a298-63765bdfcd06 tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:30:09.150668 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-87bcd5e2-52a0-4d37-a298-63765bdfcd06 tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:30:09.151165 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-87bcd5e2-52a0-4d37-a298-63765bdfcd06 tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:30:09.385522 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-87bcd5e2-52a0-4d37-a298-63765bdfcd06 tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '642291c4daa04a6ab3fbce92a8be39c6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b2ddff853b7546788b645a0344c2297b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:30:09.396570 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-87bcd5e2-52a0-4d37-a298-63765bdfcd06 tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] 10.222.0.52 "GET /compute/v2.1/servers/9941c49f-8626-461d-a570-c0ffbfd4b804" status: 200 len: 1576 microversion: 2.1 time: 0.260185 Aug 27 02:30:09.397170 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 392/789] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:30:09 2022] GET /compute/v2.1/servers/9941c49f-8626-461d-a570-c0ffbfd4b804 => generated 1576 bytes in 262 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:30:09.504139 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.virt.hardware [None req-af499363-fa18-4de9-9511-8f6400655ba5 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Flavor limits 0:0:0 {{(pid=120864) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Aug 27 02:30:09.504646 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.virt.hardware [None req-af499363-fa18-4de9-9511-8f6400655ba5 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Image limits 0:0:0 {{(pid=120864) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Aug 27 02:30:09.505045 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.virt.hardware [None req-af499363-fa18-4de9-9511-8f6400655ba5 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Flavor pref 0:0:0 {{(pid=120864) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Aug 27 02:30:09.505481 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.virt.hardware [None req-af499363-fa18-4de9-9511-8f6400655ba5 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Image pref 0:0:0 {{(pid=120864) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Aug 27 02:30:09.505886 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.virt.hardware [None req-af499363-fa18-4de9-9511-8f6400655ba5 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=120864) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Aug 27 02:30:09.515494 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-af499363-fa18-4de9-9511-8f6400655ba5 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:30:09.515968 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-af499363-fa18-4de9-9511-8f6400655ba5 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:30:09.516394 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-af499363-fa18-4de9-9511-8f6400655ba5 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:30:09.517611 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-af499363-fa18-4de9-9511-8f6400655ba5 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:30:09.517878 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-af499363-fa18-4de9-9511-8f6400655ba5 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:30:09.518295 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-af499363-fa18-4de9-9511-8f6400655ba5 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:30:09.536053 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.quota [None req-af499363-fa18-4de9-9511-8f6400655ba5 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Getting quotas for project 5d5ec8450c0941d18d63558baab54737. Resources: {'cores', 'ram', 'instances'} {{(pid=120864) _get_quotas /opt/stack/nova/nova/quota.py:393}} Aug 27 02:30:09.541746 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.quota [None req-af499363-fa18-4de9-9511-8f6400655ba5 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Getting quotas for user 26a86351583d49d090b3db09f127d49f and project 5d5ec8450c0941d18d63558baab54737. Resources: {'cores', 'ram', 'instances'} {{(pid=120864) _get_quotas /opt/stack/nova/nova/quota.py:383}} Aug 27 02:30:09.543531 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG dbcounter [-] [120864] Writing DB stats nova_api:SELECT=96,nova_api:UPDATE=2,nova_api:DELETE=2 {{(pid=120864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 27 02:30:09.557673 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-af499363-fa18-4de9-9511-8f6400655ba5 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Going to run 1 instances... {{(pid=120864) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Aug 27 02:30:09.560063 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-af499363-fa18-4de9-9511-8f6400655ba5 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] [instance: 2ad7bd6d-558b-4b12-a22f-67f33975e5fd] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='9fc30b36-b827-46f9-892d-30e808bc2b8a',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=120864) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Aug 27 02:30:09.622524 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-af499363-fa18-4de9-9511-8f6400655ba5 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] 10.222.0.52 "POST /compute/v2.1/servers" status: 202 len: 402 microversion: 2.1 time: 0.700279 Aug 27 02:30:09.624580 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 398/790] 10.222.0.52 () {60 vars in 1147 bytes} [Sat Aug 27 02:30:08 2022] POST /compute/v2.1/servers => generated 402 bytes in 702 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Aug 27 02:30:09.643510 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-f9558348-b593-4e1b-9542-84b8ad96cdd6 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:30:09.644422 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-f9558348-b593-4e1b-9542-84b8ad96cdd6 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] [instance: 2ad7bd6d-558b-4b12-a22f-67f33975e5fd] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:30:09.714316 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-f9558348-b593-4e1b-9542-84b8ad96cdd6 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '26a86351583d49d090b3db09f127d49f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5d5ec8450c0941d18d63558baab54737', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:30:09.721563 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-f9558348-b593-4e1b-9542-84b8ad96cdd6 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] 10.222.0.52 "GET /compute/v2.1/servers/2ad7bd6d-558b-4b12-a22f-67f33975e5fd" status: 200 len: 1240 microversion: 2.1 time: 0.090140 Aug 27 02:30:09.722525 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 393/791] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:30:09 2022] GET /compute/v2.1/servers/2ad7bd6d-558b-4b12-a22f-67f33975e5fd => generated 1240 bytes in 92 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:30:09.744829 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-5103260c-cb2e-421e-83a5-c7cc9518bd2b tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Calling method '>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:30:09.745483 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-5103260c-cb2e-421e-83a5-c7cc9518bd2b tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] [instance: 98e4cfeb-d49e-4888-a2c0-40a9e8af90d3] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:30:09.752204 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-5103260c-cb2e-421e-83a5-c7cc9518bd2b tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:30:09.752715 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-5103260c-cb2e-421e-83a5-c7cc9518bd2b tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:30:09.753305 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-5103260c-cb2e-421e-83a5-c7cc9518bd2b tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:30:09.753941 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-5103260c-cb2e-421e-83a5-c7cc9518bd2b tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:30:09.754348 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-5103260c-cb2e-421e-83a5-c7cc9518bd2b tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:30:09.754781 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-5103260c-cb2e-421e-83a5-c7cc9518bd2b tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:30:09.897161 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-5103260c-cb2e-421e-83a5-c7cc9518bd2b tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd945d512ae034ec08838d0a3afde2429', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '880ac4003248441c809a76cfd1b221eb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:30:09.908789 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-5103260c-cb2e-421e-83a5-c7cc9518bd2b tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] 10.222.0.52 "GET /compute/v2.1/servers/98e4cfeb-d49e-4888-a2c0-40a9e8af90d3" status: 200 len: 1729 microversion: 2.1 time: 0.168579 Aug 27 02:30:09.912907 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 399/792] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:30:09 2022] GET /compute/v2.1/servers/98e4cfeb-d49e-4888-a2c0-40a9e8af90d3 => generated 1729 bytes in 170 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:30:09.930526 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-86f46e44-c79f-40e2-8fe4-370047cbb5ae tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:30:09.931167 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-86f46e44-c79f-40e2-8fe4-370047cbb5ae tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] [instance: 98e4cfeb-d49e-4888-a2c0-40a9e8af90d3] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:30:09.937474 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-86f46e44-c79f-40e2-8fe4-370047cbb5ae tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:30:09.938076 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-86f46e44-c79f-40e2-8fe4-370047cbb5ae tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:30:09.938522 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-86f46e44-c79f-40e2-8fe4-370047cbb5ae tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:30:09.939228 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-86f46e44-c79f-40e2-8fe4-370047cbb5ae tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:30:09.939683 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-86f46e44-c79f-40e2-8fe4-370047cbb5ae tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:30:09.940106 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-86f46e44-c79f-40e2-8fe4-370047cbb5ae tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:30:10.021947 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-86f46e44-c79f-40e2-8fe4-370047cbb5ae tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] 10.222.0.52 "GET /compute/v2.1/servers/98e4cfeb-d49e-4888-a2c0-40a9e8af90d3/os-interface" status: 200 len: 285 microversion: 2.1 time: 0.095685 Aug 27 02:30:10.022513 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 394/793] 10.222.0.52 () {58 vars in 1275 bytes} [Sat Aug 27 02:30:09 2022] GET /compute/v2.1/servers/98e4cfeb-d49e-4888-a2c0-40a9e8af90d3/os-interface => generated 285 bytes in 97 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 27 02:30:10.420470 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-323fb07d-a780-4272-b327-772b63cffab2 tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] Calling method '>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:30:10.421494 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-323fb07d-a780-4272-b327-772b63cffab2 tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] [instance: 9941c49f-8626-461d-a570-c0ffbfd4b804] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:30:10.435604 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-323fb07d-a780-4272-b327-772b63cffab2 tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:30:10.436361 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-323fb07d-a780-4272-b327-772b63cffab2 tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:30:10.437003 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-323fb07d-a780-4272-b327-772b63cffab2 tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:30:10.438109 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-323fb07d-a780-4272-b327-772b63cffab2 tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:30:10.438819 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-323fb07d-a780-4272-b327-772b63cffab2 tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:30:10.439486 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-323fb07d-a780-4272-b327-772b63cffab2 tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:30:10.620484 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-323fb07d-a780-4272-b327-772b63cffab2 tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '642291c4daa04a6ab3fbce92a8be39c6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b2ddff853b7546788b645a0344c2297b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:30:10.631066 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-323fb07d-a780-4272-b327-772b63cffab2 tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] 10.222.0.52 "GET /compute/v2.1/servers/9941c49f-8626-461d-a570-c0ffbfd4b804" status: 200 len: 1576 microversion: 2.1 time: 0.216053 Aug 27 02:30:10.631749 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 400/794] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:30:10 2022] GET /compute/v2.1/servers/9941c49f-8626-461d-a570-c0ffbfd4b804 => generated 1576 bytes in 218 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:30:10.742759 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-587a5f22-db5c-4ae0-8ad0-002dbebaa0d4 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:30:10.743331 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-587a5f22-db5c-4ae0-8ad0-002dbebaa0d4 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] [instance: 2ad7bd6d-558b-4b12-a22f-67f33975e5fd] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:30:10.749753 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-587a5f22-db5c-4ae0-8ad0-002dbebaa0d4 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:30:10.750324 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-587a5f22-db5c-4ae0-8ad0-002dbebaa0d4 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:30:10.750875 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-587a5f22-db5c-4ae0-8ad0-002dbebaa0d4 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:30:10.751687 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-587a5f22-db5c-4ae0-8ad0-002dbebaa0d4 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:30:10.752267 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-587a5f22-db5c-4ae0-8ad0-002dbebaa0d4 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:30:10.752792 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-587a5f22-db5c-4ae0-8ad0-002dbebaa0d4 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:30:10.832064 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-587a5f22-db5c-4ae0-8ad0-002dbebaa0d4 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '26a86351583d49d090b3db09f127d49f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5d5ec8450c0941d18d63558baab54737', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:30:10.841221 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-587a5f22-db5c-4ae0-8ad0-002dbebaa0d4 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] 10.222.0.52 "GET /compute/v2.1/servers/2ad7bd6d-558b-4b12-a22f-67f33975e5fd" status: 200 len: 1236 microversion: 2.1 time: 0.103223 Aug 27 02:30:10.841809 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 395/795] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:30:10 2022] GET /compute/v2.1/servers/2ad7bd6d-558b-4b12-a22f-67f33975e5fd => generated 1236 bytes in 105 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:30:11.259420 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [req-415d44ed-b2f2-4141-bc28-94b4e2120168 req-80602042-cf4b-4f0b-ac8b-1688762d9dc1 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "98e4cfeb-d49e-4888-a2c0-40a9e8af90d3", "tag": "bf498aa0-7dfa-466b-b4f0-3a5d8f8703e2"}]} {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 27 02:30:11.273336 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [req-415d44ed-b2f2-4141-bc28-94b4e2120168 req-80602042-cf4b-4f0b-ac8b-1688762d9dc1 service nova] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:30:11.274797 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [req-415d44ed-b2f2-4141-bc28-94b4e2120168 req-80602042-cf4b-4f0b-ac8b-1688762d9dc1 service nova] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:30:11.276015 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [req-415d44ed-b2f2-4141-bc28-94b4e2120168 req-80602042-cf4b-4f0b-ac8b-1688762d9dc1 service nova] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:30:11.291302 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.compute.server_external_events [req-415d44ed-b2f2-4141-bc28-94b4e2120168 req-80602042-cf4b-4f0b-ac8b-1688762d9dc1 service nova] Creating event network-changed:bf498aa0-7dfa-466b-b4f0-3a5d8f8703e2 for instance 98e4cfeb-d49e-4888-a2c0-40a9e8af90d3 on hv-cinder-84630 Aug 27 02:30:11.296054 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [req-415d44ed-b2f2-4141-bc28-94b4e2120168 req-80602042-cf4b-4f0b-ac8b-1688762d9dc1 service nova] 10.222.0.52 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.041639 Aug 27 02:30:11.296618 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 401/796] 10.222.0.52 () {64 vars in 1317 bytes} [Sat Aug 27 02:30:11 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Aug 27 02:30:11.653202 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-a34bf7b9-cdd9-4a8e-acd5-52a11ff68a61 tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:30:11.653806 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-a34bf7b9-cdd9-4a8e-acd5-52a11ff68a61 tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] [instance: 9941c49f-8626-461d-a570-c0ffbfd4b804] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:30:11.661778 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-a34bf7b9-cdd9-4a8e-acd5-52a11ff68a61 tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:30:11.662263 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-a34bf7b9-cdd9-4a8e-acd5-52a11ff68a61 tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:30:11.662709 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-a34bf7b9-cdd9-4a8e-acd5-52a11ff68a61 tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:30:11.663413 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-a34bf7b9-cdd9-4a8e-acd5-52a11ff68a61 tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:30:11.663855 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-a34bf7b9-cdd9-4a8e-acd5-52a11ff68a61 tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:30:11.664272 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-a34bf7b9-cdd9-4a8e-acd5-52a11ff68a61 tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:30:11.827135 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-a34bf7b9-cdd9-4a8e-acd5-52a11ff68a61 tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '642291c4daa04a6ab3fbce92a8be39c6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b2ddff853b7546788b645a0344c2297b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:30:11.835316 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-a34bf7b9-cdd9-4a8e-acd5-52a11ff68a61 tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] 10.222.0.52 "GET /compute/v2.1/servers/9941c49f-8626-461d-a570-c0ffbfd4b804" status: 200 len: 1563 microversion: 2.1 time: 0.185919 Aug 27 02:30:11.835924 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 396/797] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:30:11 2022] GET /compute/v2.1/servers/9941c49f-8626-461d-a570-c0ffbfd4b804 => generated 1563 bytes in 187 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:30:11.864698 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-be2f34aa-9fff-47b0-a51f-fa267fb8fa27 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Calling method '>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:30:11.865559 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-be2f34aa-9fff-47b0-a51f-fa267fb8fa27 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] [instance: 2ad7bd6d-558b-4b12-a22f-67f33975e5fd] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:30:11.872662 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-be2f34aa-9fff-47b0-a51f-fa267fb8fa27 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:30:11.873287 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-be2f34aa-9fff-47b0-a51f-fa267fb8fa27 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:30:11.873732 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-be2f34aa-9fff-47b0-a51f-fa267fb8fa27 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:30:11.874427 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-be2f34aa-9fff-47b0-a51f-fa267fb8fa27 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:30:11.874885 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-be2f34aa-9fff-47b0-a51f-fa267fb8fa27 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:30:11.875305 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-be2f34aa-9fff-47b0-a51f-fa267fb8fa27 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:30:11.945103 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-be2f34aa-9fff-47b0-a51f-fa267fb8fa27 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '26a86351583d49d090b3db09f127d49f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5d5ec8450c0941d18d63558baab54737', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:30:11.957477 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-be2f34aa-9fff-47b0-a51f-fa267fb8fa27 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] 10.222.0.52 "GET /compute/v2.1/servers/2ad7bd6d-558b-4b12-a22f-67f33975e5fd" status: 200 len: 1292 microversion: 2.1 time: 0.097648 Aug 27 02:30:11.958441 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 402/798] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:30:11 2022] GET /compute/v2.1/servers/2ad7bd6d-558b-4b12-a22f-67f33975e5fd => generated 1292 bytes in 99 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:30:12.983088 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-f9a393bf-1461-4d8c-b9e9-65d365a18d72 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:30:12.984299 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-f9a393bf-1461-4d8c-b9e9-65d365a18d72 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] [instance: 2ad7bd6d-558b-4b12-a22f-67f33975e5fd] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:30:12.987073 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG dbcounter [-] [120863] Writing DB stats nova_api:SELECT=98,nova_api:UPDATE=1,nova_api:INSERT=1 {{(pid=120863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 27 02:30:12.994548 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-f9a393bf-1461-4d8c-b9e9-65d365a18d72 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:30:12.995026 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-f9a393bf-1461-4d8c-b9e9-65d365a18d72 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:30:12.995644 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-f9a393bf-1461-4d8c-b9e9-65d365a18d72 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:30:12.996545 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-f9a393bf-1461-4d8c-b9e9-65d365a18d72 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:30:12.997558 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-f9a393bf-1461-4d8c-b9e9-65d365a18d72 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:30:12.997661 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-f9a393bf-1461-4d8c-b9e9-65d365a18d72 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:30:13.261456 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-f9a393bf-1461-4d8c-b9e9-65d365a18d72 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '26a86351583d49d090b3db09f127d49f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5d5ec8450c0941d18d63558baab54737', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:30:13.270357 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-f9a393bf-1461-4d8c-b9e9-65d365a18d72 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] 10.222.0.52 "GET /compute/v2.1/servers/2ad7bd6d-558b-4b12-a22f-67f33975e5fd" status: 200 len: 1366 microversion: 2.1 time: 0.294800 Aug 27 02:30:13.270955 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 397/799] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:30:12 2022] GET /compute/v2.1/servers/2ad7bd6d-558b-4b12-a22f-67f33975e5fd => generated 1366 bytes in 297 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:30:13.310933 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [req-5fa474cd-62dd-4c52-a078-bb4870e41bab req-90b7d2ef-fa7d-4135-8f20-dd55200b231c service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "98e4cfeb-d49e-4888-a2c0-40a9e8af90d3", "tag": "bf498aa0-7dfa-466b-b4f0-3a5d8f8703e2"}]} {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 27 02:30:13.326495 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [req-5fa474cd-62dd-4c52-a078-bb4870e41bab req-90b7d2ef-fa7d-4135-8f20-dd55200b231c service nova] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:30:13.327005 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [req-5fa474cd-62dd-4c52-a078-bb4870e41bab req-90b7d2ef-fa7d-4135-8f20-dd55200b231c service nova] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:30:13.327446 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [req-5fa474cd-62dd-4c52-a078-bb4870e41bab req-90b7d2ef-fa7d-4135-8f20-dd55200b231c service nova] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:30:13.341725 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.compute.server_external_events [req-5fa474cd-62dd-4c52-a078-bb4870e41bab req-90b7d2ef-fa7d-4135-8f20-dd55200b231c service nova] Creating event network-changed:bf498aa0-7dfa-466b-b4f0-3a5d8f8703e2 for instance 98e4cfeb-d49e-4888-a2c0-40a9e8af90d3 on hv-cinder-84630 Aug 27 02:30:13.346910 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [req-5fa474cd-62dd-4c52-a078-bb4870e41bab req-90b7d2ef-fa7d-4135-8f20-dd55200b231c service nova] 10.222.0.52 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.039603 Aug 27 02:30:13.347508 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 403/800] 10.222.0.52 () {64 vars in 1317 bytes} [Sat Aug 27 02:30:13 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 41 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 27 02:30:13.613049 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [req-9c043970-aff7-431f-9892-2d140fe326fc req-143db6fa-78d0-47a9-a13b-0ac31a6bef26 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "2ad7bd6d-558b-4b12-a22f-67f33975e5fd", "tag": "d49ef841-e8bf-49f7-b85b-a7fbc34a1e23"}]} {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 27 02:30:13.625489 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [req-9c043970-aff7-431f-9892-2d140fe326fc req-143db6fa-78d0-47a9-a13b-0ac31a6bef26 service nova] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:30:13.626077 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [req-9c043970-aff7-431f-9892-2d140fe326fc req-143db6fa-78d0-47a9-a13b-0ac31a6bef26 service nova] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:30:13.626578 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [req-9c043970-aff7-431f-9892-2d140fe326fc req-143db6fa-78d0-47a9-a13b-0ac31a6bef26 service nova] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:30:13.638777 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.compute.server_external_events [req-9c043970-aff7-431f-9892-2d140fe326fc req-143db6fa-78d0-47a9-a13b-0ac31a6bef26 service nova] Creating event network-changed:d49ef841-e8bf-49f7-b85b-a7fbc34a1e23 for instance 2ad7bd6d-558b-4b12-a22f-67f33975e5fd on hv-cinder-84630 Aug 27 02:30:13.644288 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [req-9c043970-aff7-431f-9892-2d140fe326fc req-143db6fa-78d0-47a9-a13b-0ac31a6bef26 service nova] 10.222.0.52 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.034782 Aug 27 02:30:13.644888 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 398/801] 10.222.0.52 () {64 vars in 1317 bytes} [Sat Aug 27 02:30:13 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 36 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 27 02:30:14.296609 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-e3fc8d7f-f170-40dc-b9aa-6e100d6ff14b tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Calling method '>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:30:14.297563 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-e3fc8d7f-f170-40dc-b9aa-6e100d6ff14b tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] [instance: 2ad7bd6d-558b-4b12-a22f-67f33975e5fd] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:30:14.307925 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-e3fc8d7f-f170-40dc-b9aa-6e100d6ff14b tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:30:14.308637 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-e3fc8d7f-f170-40dc-b9aa-6e100d6ff14b tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:30:14.309386 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-e3fc8d7f-f170-40dc-b9aa-6e100d6ff14b tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:30:14.310414 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-e3fc8d7f-f170-40dc-b9aa-6e100d6ff14b tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:30:14.311169 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-e3fc8d7f-f170-40dc-b9aa-6e100d6ff14b tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:30:14.312153 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-e3fc8d7f-f170-40dc-b9aa-6e100d6ff14b tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:30:14.468771 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-e3fc8d7f-f170-40dc-b9aa-6e100d6ff14b tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '26a86351583d49d090b3db09f127d49f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5d5ec8450c0941d18d63558baab54737', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:30:14.479065 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-e3fc8d7f-f170-40dc-b9aa-6e100d6ff14b tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] 10.222.0.52 "GET /compute/v2.1/servers/2ad7bd6d-558b-4b12-a22f-67f33975e5fd" status: 200 len: 1366 microversion: 2.1 time: 0.189591 Aug 27 02:30:14.479656 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 404/802] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:30:14 2022] GET /compute/v2.1/servers/2ad7bd6d-558b-4b12-a22f-67f33975e5fd => generated 1366 bytes in 191 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:30:15.500820 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-f39a3083-afdf-45d5-b7fe-ef16ba124a1d tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:30:15.501573 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-f39a3083-afdf-45d5-b7fe-ef16ba124a1d tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] [instance: 2ad7bd6d-558b-4b12-a22f-67f33975e5fd] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:30:15.508913 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-f39a3083-afdf-45d5-b7fe-ef16ba124a1d tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:30:15.509511 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-f39a3083-afdf-45d5-b7fe-ef16ba124a1d tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:30:15.509979 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-f39a3083-afdf-45d5-b7fe-ef16ba124a1d tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:30:15.510700 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-f39a3083-afdf-45d5-b7fe-ef16ba124a1d tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:30:15.511157 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-f39a3083-afdf-45d5-b7fe-ef16ba124a1d tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:30:15.511578 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-f39a3083-afdf-45d5-b7fe-ef16ba124a1d tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:30:15.663407 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-f39a3083-afdf-45d5-b7fe-ef16ba124a1d tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '26a86351583d49d090b3db09f127d49f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5d5ec8450c0941d18d63558baab54737', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:30:15.672196 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-f39a3083-afdf-45d5-b7fe-ef16ba124a1d tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] 10.222.0.52 "GET /compute/v2.1/servers/2ad7bd6d-558b-4b12-a22f-67f33975e5fd" status: 200 len: 1366 microversion: 2.1 time: 0.176130 Aug 27 02:30:15.672766 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 399/803] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:30:15 2022] GET /compute/v2.1/servers/2ad7bd6d-558b-4b12-a22f-67f33975e5fd => generated 1366 bytes in 178 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:30:16.694602 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-dcdb48d0-8d99-435f-b03b-f2664d94ff0d tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Calling method '>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:30:16.695184 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-dcdb48d0-8d99-435f-b03b-f2664d94ff0d tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] [instance: 2ad7bd6d-558b-4b12-a22f-67f33975e5fd] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:30:16.704229 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-dcdb48d0-8d99-435f-b03b-f2664d94ff0d tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:30:16.704719 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-dcdb48d0-8d99-435f-b03b-f2664d94ff0d tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:30:16.705147 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-dcdb48d0-8d99-435f-b03b-f2664d94ff0d tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:30:16.705850 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-dcdb48d0-8d99-435f-b03b-f2664d94ff0d tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:30:16.706298 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-dcdb48d0-8d99-435f-b03b-f2664d94ff0d tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:30:16.706727 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-dcdb48d0-8d99-435f-b03b-f2664d94ff0d tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:30:16.863976 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-dcdb48d0-8d99-435f-b03b-f2664d94ff0d tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '26a86351583d49d090b3db09f127d49f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5d5ec8450c0941d18d63558baab54737', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:30:16.873945 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-dcdb48d0-8d99-435f-b03b-f2664d94ff0d tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] 10.222.0.52 "GET /compute/v2.1/servers/2ad7bd6d-558b-4b12-a22f-67f33975e5fd" status: 200 len: 1366 microversion: 2.1 time: 0.183580 Aug 27 02:30:16.874516 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 405/804] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:30:16 2022] GET /compute/v2.1/servers/2ad7bd6d-558b-4b12-a22f-67f33975e5fd => generated 1366 bytes in 185 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:30:17.895835 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-4922b701-1719-4949-bdf5-baadae1df2f6 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:30:17.896439 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-4922b701-1719-4949-bdf5-baadae1df2f6 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] [instance: 2ad7bd6d-558b-4b12-a22f-67f33975e5fd] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:30:17.903931 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-4922b701-1719-4949-bdf5-baadae1df2f6 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:30:17.904453 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-4922b701-1719-4949-bdf5-baadae1df2f6 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:30:17.904914 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-4922b701-1719-4949-bdf5-baadae1df2f6 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:30:17.905655 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-4922b701-1719-4949-bdf5-baadae1df2f6 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:30:17.906115 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-4922b701-1719-4949-bdf5-baadae1df2f6 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:30:17.906544 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-4922b701-1719-4949-bdf5-baadae1df2f6 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:30:18.049286 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-4922b701-1719-4949-bdf5-baadae1df2f6 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '26a86351583d49d090b3db09f127d49f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5d5ec8450c0941d18d63558baab54737', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:30:18.058545 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-4922b701-1719-4949-bdf5-baadae1df2f6 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] 10.222.0.52 "GET /compute/v2.1/servers/2ad7bd6d-558b-4b12-a22f-67f33975e5fd" status: 200 len: 1366 microversion: 2.1 time: 0.167198 Aug 27 02:30:18.059137 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 400/805] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:30:17 2022] GET /compute/v2.1/servers/2ad7bd6d-558b-4b12-a22f-67f33975e5fd => generated 1366 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:30:19.084487 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-a439b747-033e-44dc-be60-6703b9860e18 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Calling method '>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:30:19.085275 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-a439b747-033e-44dc-be60-6703b9860e18 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] [instance: 2ad7bd6d-558b-4b12-a22f-67f33975e5fd] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:30:19.092960 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-a439b747-033e-44dc-be60-6703b9860e18 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:30:19.093468 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-a439b747-033e-44dc-be60-6703b9860e18 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:30:19.093894 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-a439b747-033e-44dc-be60-6703b9860e18 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:30:19.094584 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-a439b747-033e-44dc-be60-6703b9860e18 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:30:19.095027 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-a439b747-033e-44dc-be60-6703b9860e18 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:30:19.095481 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-a439b747-033e-44dc-be60-6703b9860e18 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:30:19.235613 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-a439b747-033e-44dc-be60-6703b9860e18 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '26a86351583d49d090b3db09f127d49f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5d5ec8450c0941d18d63558baab54737', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:30:19.243741 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-a439b747-033e-44dc-be60-6703b9860e18 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] 10.222.0.52 "GET /compute/v2.1/servers/2ad7bd6d-558b-4b12-a22f-67f33975e5fd" status: 200 len: 1366 microversion: 2.1 time: 0.164702 Aug 27 02:30:19.244320 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 406/806] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:30:19 2022] GET /compute/v2.1/servers/2ad7bd6d-558b-4b12-a22f-67f33975e5fd => generated 1366 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:30:20.089444 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [req-79743359-df4a-424c-9228-f72343401995 req-fa0a0451-91da-40bf-8ec2-f481f81e58a6 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "2ad7bd6d-558b-4b12-a22f-67f33975e5fd", "name": "network-vif-plugged", "status": "completed", "tag": "d49ef841-e8bf-49f7-b85b-a7fbc34a1e23"}]} {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 27 02:30:20.095428 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG dbcounter [-] [120863] Writing DB stats nova_cell0:SELECT=6 {{(pid=120863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 27 02:30:20.101768 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [req-79743359-df4a-424c-9228-f72343401995 req-fa0a0451-91da-40bf-8ec2-f481f81e58a6 service nova] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:30:20.102498 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [req-79743359-df4a-424c-9228-f72343401995 req-fa0a0451-91da-40bf-8ec2-f481f81e58a6 service nova] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:30:20.104058 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [req-79743359-df4a-424c-9228-f72343401995 req-fa0a0451-91da-40bf-8ec2-f481f81e58a6 service nova] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:30:20.116568 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.compute.server_external_events [req-79743359-df4a-424c-9228-f72343401995 req-fa0a0451-91da-40bf-8ec2-f481f81e58a6 service nova] Creating event network-vif-plugged:d49ef841-e8bf-49f7-b85b-a7fbc34a1e23 for instance 2ad7bd6d-558b-4b12-a22f-67f33975e5fd on hv-cinder-84630 Aug 27 02:30:20.122233 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [req-79743359-df4a-424c-9228-f72343401995 req-fa0a0451-91da-40bf-8ec2-f481f81e58a6 service nova] 10.222.0.52 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.037793 Aug 27 02:30:20.123202 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 401/807] 10.222.0.52 () {64 vars in 1317 bytes} [Sat Aug 27 02:30:20 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 40 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 27 02:30:20.265469 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-acfc4b43-bb48-4061-bd3a-4231a54143c1 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Calling method '>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:30:20.266007 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-acfc4b43-bb48-4061-bd3a-4231a54143c1 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] [instance: 2ad7bd6d-558b-4b12-a22f-67f33975e5fd] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:30:20.267987 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG dbcounter [-] [120864] Writing DB stats nova_cell0:SELECT=2 {{(pid=120864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 27 02:30:20.268652 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG dbcounter [-] [120864] Writing DB stats nova_cell0:SELECT=6 {{(pid=120864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 27 02:30:20.276649 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-acfc4b43-bb48-4061-bd3a-4231a54143c1 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:30:20.277154 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-acfc4b43-bb48-4061-bd3a-4231a54143c1 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:30:20.277582 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-acfc4b43-bb48-4061-bd3a-4231a54143c1 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:30:20.278280 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-acfc4b43-bb48-4061-bd3a-4231a54143c1 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:30:20.278719 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-acfc4b43-bb48-4061-bd3a-4231a54143c1 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:30:20.279190 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-acfc4b43-bb48-4061-bd3a-4231a54143c1 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:30:20.489987 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-acfc4b43-bb48-4061-bd3a-4231a54143c1 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '26a86351583d49d090b3db09f127d49f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5d5ec8450c0941d18d63558baab54737', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:30:20.498449 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-acfc4b43-bb48-4061-bd3a-4231a54143c1 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] 10.222.0.52 "GET /compute/v2.1/servers/2ad7bd6d-558b-4b12-a22f-67f33975e5fd" status: 200 len: 1366 microversion: 2.1 time: 0.237027 Aug 27 02:30:20.499026 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 407/808] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:30:20 2022] GET /compute/v2.1/servers/2ad7bd6d-558b-4b12-a22f-67f33975e5fd => generated 1366 bytes in 238 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:30:21.522376 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-5da2f564-ebb3-4a4b-9fea-7276c586c6c1 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:30:21.523135 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-5da2f564-ebb3-4a4b-9fea-7276c586c6c1 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] [instance: 2ad7bd6d-558b-4b12-a22f-67f33975e5fd] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:30:21.531883 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-5da2f564-ebb3-4a4b-9fea-7276c586c6c1 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:30:21.532544 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-5da2f564-ebb3-4a4b-9fea-7276c586c6c1 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:30:21.532899 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-5da2f564-ebb3-4a4b-9fea-7276c586c6c1 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:30:21.533876 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-5da2f564-ebb3-4a4b-9fea-7276c586c6c1 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:30:21.533876 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-5da2f564-ebb3-4a4b-9fea-7276c586c6c1 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:30:21.534595 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-5da2f564-ebb3-4a4b-9fea-7276c586c6c1 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:30:21.672825 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-5da2f564-ebb3-4a4b-9fea-7276c586c6c1 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '26a86351583d49d090b3db09f127d49f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5d5ec8450c0941d18d63558baab54737', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:30:21.680794 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-5da2f564-ebb3-4a4b-9fea-7276c586c6c1 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] 10.222.0.52 "GET /compute/v2.1/servers/2ad7bd6d-558b-4b12-a22f-67f33975e5fd" status: 200 len: 1366 microversion: 2.1 time: 0.164606 Aug 27 02:30:21.681393 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 402/809] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:30:21 2022] GET /compute/v2.1/servers/2ad7bd6d-558b-4b12-a22f-67f33975e5fd => generated 1366 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:30:22.701913 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-fce98aa7-5102-4775-b73a-0f6a9f1d2555 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Calling method '>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:30:22.702544 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-fce98aa7-5102-4775-b73a-0f6a9f1d2555 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] [instance: 2ad7bd6d-558b-4b12-a22f-67f33975e5fd] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:30:22.708937 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-fce98aa7-5102-4775-b73a-0f6a9f1d2555 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:30:22.709442 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-fce98aa7-5102-4775-b73a-0f6a9f1d2555 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:30:22.709878 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-fce98aa7-5102-4775-b73a-0f6a9f1d2555 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:30:22.710543 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-fce98aa7-5102-4775-b73a-0f6a9f1d2555 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:30:22.711178 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-fce98aa7-5102-4775-b73a-0f6a9f1d2555 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:30:22.712164 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-fce98aa7-5102-4775-b73a-0f6a9f1d2555 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:30:22.865046 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-fce98aa7-5102-4775-b73a-0f6a9f1d2555 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '26a86351583d49d090b3db09f127d49f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5d5ec8450c0941d18d63558baab54737', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:30:22.874652 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-fce98aa7-5102-4775-b73a-0f6a9f1d2555 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] 10.222.0.52 "GET /compute/v2.1/servers/2ad7bd6d-558b-4b12-a22f-67f33975e5fd" status: 200 len: 1556 microversion: 2.1 time: 0.176608 Aug 27 02:30:22.875221 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 408/810] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:30:22 2022] GET /compute/v2.1/servers/2ad7bd6d-558b-4b12-a22f-67f33975e5fd => generated 1556 bytes in 178 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:30:22.900391 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-cc39e426-5996-4a04-9957-95548dd5f992 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:30:22.900988 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-cc39e426-5996-4a04-9957-95548dd5f992 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] [instance: 2ad7bd6d-558b-4b12-a22f-67f33975e5fd] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:30:22.907687 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-cc39e426-5996-4a04-9957-95548dd5f992 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:30:22.908175 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-cc39e426-5996-4a04-9957-95548dd5f992 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:30:22.908613 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-cc39e426-5996-4a04-9957-95548dd5f992 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:30:22.909360 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-cc39e426-5996-4a04-9957-95548dd5f992 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:30:22.909812 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-cc39e426-5996-4a04-9957-95548dd5f992 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:30:22.910213 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-cc39e426-5996-4a04-9957-95548dd5f992 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:30:22.991781 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-cc39e426-5996-4a04-9957-95548dd5f992 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] 10.222.0.52 "GET /compute/v2.1/servers/2ad7bd6d-558b-4b12-a22f-67f33975e5fd/os-interface" status: 200 len: 285 microversion: 2.1 time: 0.096513 Aug 27 02:30:22.993761 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 403/811] 10.222.0.52 () {58 vars in 1275 bytes} [Sat Aug 27 02:30:22 2022] GET /compute/v2.1/servers/2ad7bd6d-558b-4b12-a22f-67f33975e5fd/os-interface => generated 285 bytes in 99 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 27 02:30:24.006750 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [req-0da6268f-0bd8-47bf-98c1-d9eee6cc64b0 req-a8617145-15d3-400e-a086-69c4a89c61a0 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "2ad7bd6d-558b-4b12-a22f-67f33975e5fd", "tag": "d49ef841-e8bf-49f7-b85b-a7fbc34a1e23"}]} {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 27 02:30:24.016735 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [req-0da6268f-0bd8-47bf-98c1-d9eee6cc64b0 req-a8617145-15d3-400e-a086-69c4a89c61a0 service nova] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:30:24.017262 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [req-0da6268f-0bd8-47bf-98c1-d9eee6cc64b0 req-a8617145-15d3-400e-a086-69c4a89c61a0 service nova] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:30:24.017799 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [req-0da6268f-0bd8-47bf-98c1-d9eee6cc64b0 req-a8617145-15d3-400e-a086-69c4a89c61a0 service nova] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:30:24.029705 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.compute.server_external_events [req-0da6268f-0bd8-47bf-98c1-d9eee6cc64b0 req-a8617145-15d3-400e-a086-69c4a89c61a0 service nova] Creating event network-changed:d49ef841-e8bf-49f7-b85b-a7fbc34a1e23 for instance 2ad7bd6d-558b-4b12-a22f-67f33975e5fd on hv-cinder-84630 Aug 27 02:30:24.035360 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [req-0da6268f-0bd8-47bf-98c1-d9eee6cc64b0 req-a8617145-15d3-400e-a086-69c4a89c61a0 service nova] 10.222.0.52 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.032264 Aug 27 02:30:24.035979 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 409/812] 10.222.0.52 () {64 vars in 1317 bytes} [Sat Aug 27 02:30:24 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 34 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 27 02:30:59.943037 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG dbcounter [-] [120863] Writing DB stats nova_api:SELECT=13 {{(pid=120863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 27 02:30:59.943903 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG dbcounter [-] [120863] Writing DB stats nova_cell1:SELECT=73 {{(pid=120863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 27 02:30:59.949453 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-fcf10121-43f1-4055-9bc8-2aa497db5542 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:30:59.949952 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-fcf10121-43f1-4055-9bc8-2aa497db5542 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] [instance: 56dbfa6a-5b6f-4b82-977b-af735e6a6ac9] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:30:59.956877 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-fcf10121-43f1-4055-9bc8-2aa497db5542 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:30:59.957410 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-fcf10121-43f1-4055-9bc8-2aa497db5542 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:30:59.957857 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-fcf10121-43f1-4055-9bc8-2aa497db5542 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:30:59.958553 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-fcf10121-43f1-4055-9bc8-2aa497db5542 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:30:59.959028 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-fcf10121-43f1-4055-9bc8-2aa497db5542 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:30:59.959435 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-fcf10121-43f1-4055-9bc8-2aa497db5542 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:30:59.989888 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-fcf10121-43f1-4055-9bc8-2aa497db5542 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] 10.222.0.52 "GET /compute/v2.1/servers/56dbfa6a-5b6f-4b82-977b-af735e6a6ac9/os-volume_attachments/21b21458-c6cc-4e06-9522-5a468fb00a22" status: 200 len: 194 microversion: 2.20 time: 0.048560 Aug 27 02:30:59.990384 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 404/813] 10.222.0.52 () {60 vars in 1454 bytes} [Sat Aug 27 02:30:59 2022] GET /compute/v2.1/servers/56dbfa6a-5b6f-4b82-977b-af735e6a6ac9/os-volume_attachments/21b21458-c6cc-4e06-9522-5a468fb00a22 => generated 194 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Aug 27 02:31:00.062316 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG dbcounter [-] [120864] Writing DB stats nova_api:SELECT=25,nova_api:INSERT=3 {{(pid=120864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 27 02:31:00.063101 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG dbcounter [-] [120864] Writing DB stats nova_cell1:SELECT=81,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=120864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 27 02:31:00.067309 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-e64109c9-9430-4b97-8976-7be317636ba0 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Calling method '>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:31:00.067883 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-e64109c9-9430-4b97-8976-7be317636ba0 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] [instance: 56dbfa6a-5b6f-4b82-977b-af735e6a6ac9] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:31:00.075860 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-e64109c9-9430-4b97-8976-7be317636ba0 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:31:00.076346 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-e64109c9-9430-4b97-8976-7be317636ba0 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:31:00.076780 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-e64109c9-9430-4b97-8976-7be317636ba0 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:31:00.077486 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-e64109c9-9430-4b97-8976-7be317636ba0 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:31:00.077931 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-e64109c9-9430-4b97-8976-7be317636ba0 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:31:00.078343 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-e64109c9-9430-4b97-8976-7be317636ba0 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:31:00.178447 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.objects.instance [None req-e64109c9-9430-4b97-8976-7be317636ba0 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lazy-loading 'services' on Instance uuid 56dbfa6a-5b6f-4b82-977b-af735e6a6ac9 {{(pid=120864) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Aug 27 02:31:00.259939 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-e64109c9-9430-4b97-8976-7be317636ba0 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] 10.222.0.52 "DELETE /compute/v2.1/servers/56dbfa6a-5b6f-4b82-977b-af735e6a6ac9/os-volume_attachments/21b21458-c6cc-4e06-9522-5a468fb00a22" status: 202 len: 0 microversion: 2.20 time: 0.199069 Aug 27 02:31:00.260756 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 410/814] 10.222.0.52 () {60 vars in 1457 bytes} [Sat Aug 27 02:31:00 2022] DELETE /compute/v2.1/servers/56dbfa6a-5b6f-4b82-977b-af735e6a6ac9/os-volume_attachments/21b21458-c6cc-4e06-9522-5a468fb00a22 => generated 0 bytes in 201 msecs (HTTP/1.1 202) 9 headers in 363 bytes (1 switches on core 0) Aug 27 02:31:00.269654 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-eec3ba16-e404-47f6-a423-c488e3827c4e tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:31:00.271915 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-eec3ba16-e404-47f6-a423-c488e3827c4e tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] [instance: 56dbfa6a-5b6f-4b82-977b-af735e6a6ac9] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:31:00.278762 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-eec3ba16-e404-47f6-a423-c488e3827c4e tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:31:00.279256 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-eec3ba16-e404-47f6-a423-c488e3827c4e tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:31:00.279729 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-eec3ba16-e404-47f6-a423-c488e3827c4e tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:31:00.280444 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-eec3ba16-e404-47f6-a423-c488e3827c4e tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:31:00.280894 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-eec3ba16-e404-47f6-a423-c488e3827c4e tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:31:00.281381 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-eec3ba16-e404-47f6-a423-c488e3827c4e tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:31:00.312999 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-eec3ba16-e404-47f6-a423-c488e3827c4e tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] 10.222.0.52 "GET /compute/v2.1/servers/56dbfa6a-5b6f-4b82-977b-af735e6a6ac9/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.047504 Aug 27 02:31:00.313590 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 405/815] 10.222.0.52 () {60 vars in 1343 bytes} [Sat Aug 27 02:31:00 2022] GET /compute/v2.1/servers/56dbfa6a-5b6f-4b82-977b-af735e6a6ac9/os-volume_attachments => generated 197 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Aug 27 02:31:01.328656 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-82f83302-5af7-4ab8-ab0d-902fc4ef7a54 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Calling method '>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:31:01.331022 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-82f83302-5af7-4ab8-ab0d-902fc4ef7a54 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] [instance: 56dbfa6a-5b6f-4b82-977b-af735e6a6ac9] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:31:01.338412 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-82f83302-5af7-4ab8-ab0d-902fc4ef7a54 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:31:01.339196 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-82f83302-5af7-4ab8-ab0d-902fc4ef7a54 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:31:01.339634 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-82f83302-5af7-4ab8-ab0d-902fc4ef7a54 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:31:01.340327 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-82f83302-5af7-4ab8-ab0d-902fc4ef7a54 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:31:01.340775 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-82f83302-5af7-4ab8-ab0d-902fc4ef7a54 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:31:01.341225 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-82f83302-5af7-4ab8-ab0d-902fc4ef7a54 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:31:01.373035 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-82f83302-5af7-4ab8-ab0d-902fc4ef7a54 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] 10.222.0.52 "GET /compute/v2.1/servers/56dbfa6a-5b6f-4b82-977b-af735e6a6ac9/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.049031 Aug 27 02:31:01.373650 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 411/816] 10.222.0.52 () {60 vars in 1343 bytes} [Sat Aug 27 02:31:01 2022] GET /compute/v2.1/servers/56dbfa6a-5b6f-4b82-977b-af735e6a6ac9/os-volume_attachments => generated 197 bytes in 51 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Aug 27 02:31:02.387004 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-9fefdc09-9e96-4a59-aeab-fc4f22892689 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:31:02.389166 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-9fefdc09-9e96-4a59-aeab-fc4f22892689 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] [instance: 56dbfa6a-5b6f-4b82-977b-af735e6a6ac9] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:31:02.396279 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-9fefdc09-9e96-4a59-aeab-fc4f22892689 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:31:02.396771 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-9fefdc09-9e96-4a59-aeab-fc4f22892689 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:31:02.397243 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-9fefdc09-9e96-4a59-aeab-fc4f22892689 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:31:02.397930 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-9fefdc09-9e96-4a59-aeab-fc4f22892689 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:31:02.398389 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-9fefdc09-9e96-4a59-aeab-fc4f22892689 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:31:02.398854 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-9fefdc09-9e96-4a59-aeab-fc4f22892689 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:31:02.430992 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-9fefdc09-9e96-4a59-aeab-fc4f22892689 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] 10.222.0.52 "GET /compute/v2.1/servers/56dbfa6a-5b6f-4b82-977b-af735e6a6ac9/os-volume_attachments" status: 200 len: 25 microversion: 2.20 time: 0.047824 Aug 27 02:31:02.431785 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 406/817] 10.222.0.52 () {60 vars in 1343 bytes} [Sat Aug 27 02:31:02 2022] GET /compute/v2.1/servers/56dbfa6a-5b6f-4b82-977b-af735e6a6ac9/os-volume_attachments => generated 25 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:31:02.507461 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-26f1e248-784d-4d23-ab8a-7c45ee3063d0 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Calling method '>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:31:02.508032 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-26f1e248-784d-4d23-ab8a-7c45ee3063d0 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] [instance: 56dbfa6a-5b6f-4b82-977b-af735e6a6ac9] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:31:02.518539 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-26f1e248-784d-4d23-ab8a-7c45ee3063d0 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:31:02.519210 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-26f1e248-784d-4d23-ab8a-7c45ee3063d0 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:31:02.519784 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-26f1e248-784d-4d23-ab8a-7c45ee3063d0 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:31:02.520812 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-26f1e248-784d-4d23-ab8a-7c45ee3063d0 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:31:02.521520 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-26f1e248-784d-4d23-ab8a-7c45ee3063d0 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:31:02.521999 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-26f1e248-784d-4d23-ab8a-7c45ee3063d0 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:31:02.551988 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-26f1e248-784d-4d23-ab8a-7c45ee3063d0 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] [instance: 56dbfa6a-5b6f-4b82-977b-af735e6a6ac9] Going to try to terminate instance {{(pid=120864) delete /opt/stack/nova/nova/compute/api.py:2719}} Aug 27 02:31:02.636979 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-26f1e248-784d-4d23-ab8a-7c45ee3063d0 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] 10.222.0.52 "DELETE /compute/v2.1/servers/56dbfa6a-5b6f-4b82-977b-af735e6a6ac9" status: 204 len: 0 microversion: 2.20 time: 0.135141 Aug 27 02:31:02.637520 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 412/818] 10.222.0.52 () {60 vars in 1280 bytes} [Sat Aug 27 02:31:02 2022] DELETE /compute/v2.1/servers/56dbfa6a-5b6f-4b82-977b-af735e6a6ac9 => generated 0 bytes in 137 msecs (HTTP/1.1 204) 9 headers in 365 bytes (1 switches on core 0) Aug 27 02:31:02.648637 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-a3ada1a1-5a31-42f9-9078-762f236ae3e2 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:31:02.649519 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-a3ada1a1-5a31-42f9-9078-762f236ae3e2 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] [instance: 56dbfa6a-5b6f-4b82-977b-af735e6a6ac9] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:31:02.656108 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-a3ada1a1-5a31-42f9-9078-762f236ae3e2 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:31:02.656589 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-a3ada1a1-5a31-42f9-9078-762f236ae3e2 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:31:02.657017 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-a3ada1a1-5a31-42f9-9078-762f236ae3e2 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:31:02.657729 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-a3ada1a1-5a31-42f9-9078-762f236ae3e2 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:31:02.658184 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-a3ada1a1-5a31-42f9-9078-762f236ae3e2 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:31:02.658591 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-a3ada1a1-5a31-42f9-9078-762f236ae3e2 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:31:02.804365 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-a3ada1a1-5a31-42f9-9078-762f236ae3e2 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:31:02.813463 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-a3ada1a1-5a31-42f9-9078-762f236ae3e2 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:31:02.814889 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-a3ada1a1-5a31-42f9-9078-762f236ae3e2 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:31:02.816979 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-a3ada1a1-5a31-42f9-9078-762f236ae3e2 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] 10.222.0.52 "GET /compute/v2.1/servers/56dbfa6a-5b6f-4b82-977b-af735e6a6ac9" status: 200 len: 1724 microversion: 2.20 time: 0.173478 Aug 27 02:31:02.816979 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 407/819] 10.222.0.52 () {60 vars in 1277 bytes} [Sat Aug 27 02:31:02 2022] GET /compute/v2.1/servers/56dbfa6a-5b6f-4b82-977b-af735e6a6ac9 => generated 1724 bytes in 175 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 27 02:31:03.844299 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-795ff268-415e-4502-ab13-19c6552cc113 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Calling method '>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:31:03.845246 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-795ff268-415e-4502-ab13-19c6552cc113 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] [instance: 56dbfa6a-5b6f-4b82-977b-af735e6a6ac9] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:31:03.855073 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-795ff268-415e-4502-ab13-19c6552cc113 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:31:03.855553 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-795ff268-415e-4502-ab13-19c6552cc113 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:31:03.855983 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-795ff268-415e-4502-ab13-19c6552cc113 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:31:03.856680 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-795ff268-415e-4502-ab13-19c6552cc113 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:31:03.857254 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-795ff268-415e-4502-ab13-19c6552cc113 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:31:03.857583 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-795ff268-415e-4502-ab13-19c6552cc113 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:31:04.011461 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-795ff268-415e-4502-ab13-19c6552cc113 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:31:04.024469 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-795ff268-415e-4502-ab13-19c6552cc113 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:31:04.026425 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-795ff268-415e-4502-ab13-19c6552cc113 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:31:04.027591 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-795ff268-415e-4502-ab13-19c6552cc113 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] 10.222.0.52 "GET /compute/v2.1/servers/56dbfa6a-5b6f-4b82-977b-af735e6a6ac9" status: 200 len: 1724 microversion: 2.20 time: 0.190830 Aug 27 02:31:04.028153 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 413/820] 10.222.0.52 () {60 vars in 1277 bytes} [Sat Aug 27 02:31:03 2022] GET /compute/v2.1/servers/56dbfa6a-5b6f-4b82-977b-af735e6a6ac9 => generated 1724 bytes in 193 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 27 02:31:05.060533 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-1c8c80b1-da02-44d8-bc74-7b8a6e1d23a5 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:31:05.061434 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-1c8c80b1-da02-44d8-bc74-7b8a6e1d23a5 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] [instance: 56dbfa6a-5b6f-4b82-977b-af735e6a6ac9] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:31:05.064664 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Aug 27 02:31:05.064868 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: Sat Aug 27 02:31:05 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/56dbfa6a-5b6f-4b82-977b-af735e6a6ac9 (ip 10.222.0.52) !!! Aug 27 02:31:05.080126 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-1c8c80b1-da02-44d8-bc74-7b8a6e1d23a5 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:31:05.080729 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-1c8c80b1-da02-44d8-bc74-7b8a6e1d23a5 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:31:05.081226 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-1c8c80b1-da02-44d8-bc74-7b8a6e1d23a5 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:31:05.082279 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-1c8c80b1-da02-44d8-bc74-7b8a6e1d23a5 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:31:05.082556 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-1c8c80b1-da02-44d8-bc74-7b8a6e1d23a5 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:31:05.082999 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-1c8c80b1-da02-44d8-bc74-7b8a6e1d23a5 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:31:05.307202 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-1c8c80b1-da02-44d8-bc74-7b8a6e1d23a5 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:31:05.316695 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-1c8c80b1-da02-44d8-bc74-7b8a6e1d23a5 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:31:05.318180 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-1c8c80b1-da02-44d8-bc74-7b8a6e1d23a5 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:31:05.319378 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-1c8c80b1-da02-44d8-bc74-7b8a6e1d23a5 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] 10.222.0.52 "GET /compute/v2.1/servers/56dbfa6a-5b6f-4b82-977b-af735e6a6ac9" status: 200 len: 1724 microversion: 2.20 time: 0.264556 Aug 27 02:31:05.319939 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 408/821] 10.222.0.52 () {60 vars in 1277 bytes} [Sat Aug 27 02:31:05 2022] GET /compute/v2.1/servers/56dbfa6a-5b6f-4b82-977b-af735e6a6ac9 => generated 1724 bytes in 266 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 27 02:31:05.894070 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [req-f7cb2961-7779-4cda-897a-ae7539a396a1 req-3e203c19-7830-4a7e-b0b5-0f490ad6397d service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "56dbfa6a-5b6f-4b82-977b-af735e6a6ac9", "name": "network-vif-deleted", "tag": "520bd5fb-723a-4570-83ca-72759c588707"}]} {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 27 02:31:05.906945 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [req-f7cb2961-7779-4cda-897a-ae7539a396a1 req-3e203c19-7830-4a7e-b0b5-0f490ad6397d service nova] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:31:05.907771 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [req-f7cb2961-7779-4cda-897a-ae7539a396a1 req-3e203c19-7830-4a7e-b0b5-0f490ad6397d service nova] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:31:05.908359 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [req-f7cb2961-7779-4cda-897a-ae7539a396a1 req-3e203c19-7830-4a7e-b0b5-0f490ad6397d service nova] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:31:05.926076 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.compute.server_external_events [req-f7cb2961-7779-4cda-897a-ae7539a396a1 req-3e203c19-7830-4a7e-b0b5-0f490ad6397d service nova] Creating event network-vif-deleted:520bd5fb-723a-4570-83ca-72759c588707 for instance 56dbfa6a-5b6f-4b82-977b-af735e6a6ac9 on hv-cinder-84630 Aug 27 02:31:05.929425 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [req-f7cb2961-7779-4cda-897a-ae7539a396a1 req-3e203c19-7830-4a7e-b0b5-0f490ad6397d service nova] 10.222.0.52 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.039056 Aug 27 02:31:05.930011 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 414/822] 10.222.0.52 () {64 vars in 1317 bytes} [Sat Aug 27 02:31:05 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 41 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 27 02:31:06.344412 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-e1a34f55-e077-4a29-9213-9ca0b00b3ad4 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:31:06.345052 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-e1a34f55-e077-4a29-9213-9ca0b00b3ad4 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] [instance: 56dbfa6a-5b6f-4b82-977b-af735e6a6ac9] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:31:06.353087 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-e1a34f55-e077-4a29-9213-9ca0b00b3ad4 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:31:06.353630 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-e1a34f55-e077-4a29-9213-9ca0b00b3ad4 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:31:06.354062 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-e1a34f55-e077-4a29-9213-9ca0b00b3ad4 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:31:06.354792 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-e1a34f55-e077-4a29-9213-9ca0b00b3ad4 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:31:06.355346 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-e1a34f55-e077-4a29-9213-9ca0b00b3ad4 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:31:06.355770 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-e1a34f55-e077-4a29-9213-9ca0b00b3ad4 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:31:06.391269 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-e1a34f55-e077-4a29-9213-9ca0b00b3ad4 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:31:06.391764 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-e1a34f55-e077-4a29-9213-9ca0b00b3ad4 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:31:06.392204 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-e1a34f55-e077-4a29-9213-9ca0b00b3ad4 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:31:06.392664 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.objects.instance [None req-e1a34f55-e077-4a29-9213-9ca0b00b3ad4 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lazy-loading 'fault' on Instance uuid 56dbfa6a-5b6f-4b82-977b-af735e6a6ac9 {{(pid=120863) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Aug 27 02:31:06.440084 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-e1a34f55-e077-4a29-9213-9ca0b00b3ad4 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:31:06.449716 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-e1a34f55-e077-4a29-9213-9ca0b00b3ad4 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:31:06.451234 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-e1a34f55-e077-4a29-9213-9ca0b00b3ad4 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:31:06.452422 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-e1a34f55-e077-4a29-9213-9ca0b00b3ad4 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] 10.222.0.52 "GET /compute/v2.1/servers/56dbfa6a-5b6f-4b82-977b-af735e6a6ac9" status: 200 len: 1371 microversion: 2.20 time: 0.112681 Aug 27 02:31:06.453379 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 409/823] 10.222.0.52 () {60 vars in 1277 bytes} [Sat Aug 27 02:31:06 2022] GET /compute/v2.1/servers/56dbfa6a-5b6f-4b82-977b-af735e6a6ac9 => generated 1371 bytes in 114 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 27 02:31:07.479234 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-60de18b7-911b-4c42-8eab-bd23cc00c313 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Calling method '>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:31:07.480079 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-60de18b7-911b-4c42-8eab-bd23cc00c313 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] [instance: 56dbfa6a-5b6f-4b82-977b-af735e6a6ac9] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:31:07.486537 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-60de18b7-911b-4c42-8eab-bd23cc00c313 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:31:07.487022 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-60de18b7-911b-4c42-8eab-bd23cc00c313 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:31:07.487455 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-60de18b7-911b-4c42-8eab-bd23cc00c313 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:31:07.488137 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-60de18b7-911b-4c42-8eab-bd23cc00c313 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:31:07.488602 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-60de18b7-911b-4c42-8eab-bd23cc00c313 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:31:07.489023 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-60de18b7-911b-4c42-8eab-bd23cc00c313 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:31:07.523909 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-60de18b7-911b-4c42-8eab-bd23cc00c313 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:31:07.524390 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-60de18b7-911b-4c42-8eab-bd23cc00c313 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:31:07.524820 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-60de18b7-911b-4c42-8eab-bd23cc00c313 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:31:07.525328 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.objects.instance [None req-60de18b7-911b-4c42-8eab-bd23cc00c313 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lazy-loading 'fault' on Instance uuid 56dbfa6a-5b6f-4b82-977b-af735e6a6ac9 {{(pid=120864) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Aug 27 02:31:07.574715 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-60de18b7-911b-4c42-8eab-bd23cc00c313 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:31:07.583051 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-60de18b7-911b-4c42-8eab-bd23cc00c313 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:31:07.584489 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-60de18b7-911b-4c42-8eab-bd23cc00c313 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:31:07.585696 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-60de18b7-911b-4c42-8eab-bd23cc00c313 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] 10.222.0.52 "GET /compute/v2.1/servers/56dbfa6a-5b6f-4b82-977b-af735e6a6ac9" status: 200 len: 1371 microversion: 2.20 time: 0.112740 Aug 27 02:31:07.586461 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 415/824] 10.222.0.52 () {60 vars in 1277 bytes} [Sat Aug 27 02:31:07 2022] GET /compute/v2.1/servers/56dbfa6a-5b6f-4b82-977b-af735e6a6ac9 => generated 1371 bytes in 114 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 27 02:31:08.609567 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-f497b650-8447-4fe9-87f7-f023e5194b77 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:31:08.610180 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-f497b650-8447-4fe9-87f7-f023e5194b77 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] [instance: 56dbfa6a-5b6f-4b82-977b-af735e6a6ac9] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:31:08.617165 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-f497b650-8447-4fe9-87f7-f023e5194b77 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:31:08.617668 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-f497b650-8447-4fe9-87f7-f023e5194b77 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:31:08.618101 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-f497b650-8447-4fe9-87f7-f023e5194b77 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:31:08.618785 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-f497b650-8447-4fe9-87f7-f023e5194b77 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:31:08.619417 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-f497b650-8447-4fe9-87f7-f023e5194b77 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:31:08.619927 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-f497b650-8447-4fe9-87f7-f023e5194b77 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:31:08.633798 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.wsgi [None req-f497b650-8447-4fe9-87f7-f023e5194b77 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] HTTP exception thrown: Instance 56dbfa6a-5b6f-4b82-977b-af735e6a6ac9 could not be found. Aug 27 02:31:08.635486 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-f497b650-8447-4fe9-87f7-f023e5194b77 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Request method failure captured: Aug 27 02:31:08.635486 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: request: GET /compute/v2.1/servers/56dbfa6a-5b6f-4b82-977b-af735e6a6ac9 HTTP/1.1 Aug 27 02:31:08.635486 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: Accept: application/json Aug 27 02:31:08.635486 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: Accept-Encoding: identity Aug 27 02:31:08.635486 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: Connection: close Aug 27 02:31:08.635486 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: Content-Length: 0 Aug 27 02:31:08.635486 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: Content-Type: application/json Aug 27 02:31:08.635486 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: Host: 10.222.0.52 Aug 27 02:31:08.635486 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: Openstack-System-Scope: None Aug 27 02:31:08.635486 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: User-Agent: python-urllib3/1.26.11 Aug 27 02:31:08.635486 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-Auth-Token: gAAAAABjCYBiNuRGqbSfO7dm9FqSum3tt3k6TUULoLKrodG50GilNY4hLrqiBLkDm5s9EALB7jkOLV7WRs6Rnc2MYSkA5wTMgskawzqFmTTWFZKYym29LUjbwB2x6tewJf0ruhSgvMmooyOttIPsYVQZ7FYwUN1sl1AtWnQXmZA3HDwawNzSXec Aug 27 02:31:08.635486 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-Domain-Id: None Aug 27 02:31:08.635486 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-Domain-Name: None Aug 27 02:31:08.635486 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-Identity-Status: Confirmed Aug 27 02:31:08.635486 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-Is-Admin-Project: True Aug 27 02:31:08.635486 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-Openstack-Nova-Api-Version: 2.20 Aug 27 02:31:08.635486 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-Project-Domain-Id: default Aug 27 02:31:08.635486 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-Project-Domain-Name: Default Aug 27 02:31:08.635486 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-Project-Id: c938ae5fd22a43e8a01859da2a336b53 Aug 27 02:31:08.635486 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-Project-Name: tempest-AttachVolumeShelveTestJSON-363465878 Aug 27 02:31:08.635486 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-Role: reader,member Aug 27 02:31:08.635486 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-Roles: reader,member Aug 27 02:31:08.637105 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-Service-Catalog: [{"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.52:8080", "publicURL": "http://10.222.0.52:8080/v1/AUTH_c938ae5fd22a43e8a01859da2a336b53"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52:9696/networking"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/compute/v2/c938ae5fd22a43e8a01859da2a336b53"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/image"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/volume/v3/c938ae5fd22a43e8a01859da2a336b53"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/identity"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/volume/v3/c938ae5fd22a43e8a01859da2a336b53"}]}] Aug 27 02:31:08.637105 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-Tenant: tempest-AttachVolumeShelveTestJSON-363465878 Aug 27 02:31:08.637105 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-Tenant-Id: c938ae5fd22a43e8a01859da2a336b53 Aug 27 02:31:08.637105 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-Tenant-Name: tempest-AttachVolumeShelveTestJSON-363465878 Aug 27 02:31:08.637105 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-User: tempest-AttachVolumeShelveTestJSON-363465878-project Aug 27 02:31:08.637105 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-User-Domain-Id: default Aug 27 02:31:08.637105 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-User-Domain-Name: Default Aug 27 02:31:08.637105 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-User-Id: d750e6e172d34d4a9333e0c059c5442b Aug 27 02:31:08.637105 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-User-Name: tempest-AttachVolumeShelveTestJSON-363465878-project Aug 27 02:31:08.637105 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: method: > Aug 27 02:31:08.637105 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: exception: Instance 56dbfa6a-5b6f-4b82-977b-af735e6a6ac9 could not be found. Aug 27 02:31:08.637105 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: action_args: {'id': '56dbfa6a-5b6f-4b82-977b-af735e6a6ac9'} Aug 27 02:31:08.638137 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Aug 27 02:31:08.638137 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 27 02:31:08.638137 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Aug 27 02:31:08.638137 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Aug 27 02:31:08.638137 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Aug 27 02:31:08.638137 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Aug 27 02:31:08.638137 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Aug 27 02:31:08.638137 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Aug 27 02:31:08.638137 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Aug 27 02:31:08.638137 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Aug 27 02:31:08.638137 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 56dbfa6a-5b6f-4b82-977b-af735e6a6ac9 could not be found. Aug 27 02:31:08.638137 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi Aug 27 02:31:08.638137 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 27 02:31:08.638137 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi Aug 27 02:31:08.638137 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 27 02:31:08.638137 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 27 02:31:08.638137 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 27 02:31:08.638137 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Aug 27 02:31:08.638137 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Aug 27 02:31:08.638137 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Aug 27 02:31:08.639597 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Aug 27 02:31:08.639597 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Aug 27 02:31:08.639597 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi instance = self._get_server( Aug 27 02:31:08.639597 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Aug 27 02:31:08.639597 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Aug 27 02:31:08.639597 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Aug 27 02:31:08.639597 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Aug 27 02:31:08.639597 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 56dbfa6a-5b6f-4b82-977b-af735e6a6ac9 could not be found. Aug 27 02:31:08.639597 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi Aug 27 02:31:08.639597 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 27 02:31:08.639597 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi Aug 27 02:31:08.639597 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 27 02:31:08.639597 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 27 02:31:08.639597 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 27 02:31:08.639597 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Aug 27 02:31:08.639597 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Aug 27 02:31:08.639597 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 56dbfa6a-5b6f-4b82-977b-af735e6a6ac9 could not be found. Aug 27 02:31:08.639597 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi Aug 27 02:31:08.640959 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-f497b650-8447-4fe9-87f7-f023e5194b77 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Returning 404 to user: Instance 56dbfa6a-5b6f-4b82-977b-af735e6a6ac9 could not be found. {{(pid=120863) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Aug 27 02:31:08.640959 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-f497b650-8447-4fe9-87f7-f023e5194b77 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] 10.222.0.52 "GET /compute/v2.1/servers/56dbfa6a-5b6f-4b82-977b-af735e6a6ac9" status: 404 len: 111 microversion: 2.20 time: 0.031202 Aug 27 02:31:08.640959 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 410/825] 10.222.0.52 () {60 vars in 1277 bytes} [Sat Aug 27 02:31:08 2022] GET /compute/v2.1/servers/56dbfa6a-5b6f-4b82-977b-af735e6a6ac9 => generated 111 bytes in 32 msecs (HTTP/1.1 404) 9 headers in 381 bytes (1 switches on core 0) Aug 27 02:31:08.646107 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-06fd0f9c-89eb-4112-aaae-0de7dccd6646 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Calling method '.version_select at 0x7f6b1e9534c0>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:31:08.670572 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-06fd0f9c-89eb-4112-aaae-0de7dccd6646 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] 10.222.0.52 "DELETE /compute/v2.1/os-keypairs/tempest-keypair-105604837" status: 204 len: 0 microversion: 2.20 time: 0.027948 Aug 27 02:31:08.671351 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 416/826] 10.222.0.52 () {60 vars in 1259 bytes} [Sat Aug 27 02:31:08 2022] DELETE /compute/v2.1/os-keypairs/tempest-keypair-105604837 => generated 0 bytes in 30 msecs (HTTP/1.1 204) 9 headers in 365 bytes (1 switches on core 0) Aug 27 02:31:09.356183 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-a11da369-d673-441e-bf3b-dd0971b1b893 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Action: 'create', calling method: .version_select at 0x7f6b1b14a5e0>, body: {"keypair": {"name": "tempest-keypair-2023400689", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBDjI4zpWu2YpEQOGqZ1BP9xMayi60l8lVts8j+LSo03dGpj9D4TLyrPm9KulMLP3GsP1f+3K8o35IGEUUpAJJXEAO6f18e7K3eLiE28FkHKge6nv5JSIsNFOONVdZfzN4g=="}} {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 27 02:31:09.374995 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.quota [None req-a11da369-d673-441e-bf3b-dd0971b1b893 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Getting quotas for project c938ae5fd22a43e8a01859da2a336b53. Resources: {'key_pairs'} {{(pid=120863) _get_quotas /opt/stack/nova/nova/quota.py:393}} Aug 27 02:31:09.381740 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.quota [None req-a11da369-d673-441e-bf3b-dd0971b1b893 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Getting quotas for user d750e6e172d34d4a9333e0c059c5442b and project c938ae5fd22a43e8a01859da2a336b53. Resources: {'key_pairs'} {{(pid=120863) _get_quotas /opt/stack/nova/nova/quota.py:383}} Aug 27 02:31:09.410953 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-a11da369-d673-441e-bf3b-dd0971b1b893 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] 10.222.0.52 "POST /compute/v2.1/os-keypairs" status: 201 len: 401 microversion: 2.20 time: 0.059694 Aug 27 02:31:09.411592 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 411/827] 10.222.0.52 () {62 vars in 1200 bytes} [Sat Aug 27 02:31:09 2022] POST /compute/v2.1/os-keypairs => generated 401 bytes in 62 msecs (HTTP/1.1 201) 9 headers in 364 bytes (1 switches on core 0) Aug 27 02:31:10.730696 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-1ccd2ce9-ea66-4ba2-b616-dd2324a16c8f tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-AttachVolumeShelveTestJSON-server-2137411783", "imageRef": "9fc30b36-b827-46f9-892d-30e808bc2b8a", "flavorRef": "452", "adminPass": "***", "networks": [{"uuid": "daf58feb-9527-44f2-90a3-a19a525b593d"}], "security_groups": [{"name": "tempest-securitygroup--1807166467"}], "key_name": "tempest-keypair-2023400689", "user_data": "}} {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 27 02:31:10.845400 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.network.neutron [None req-1ccd2ce9-ea66-4ba2-b616-dd2324a16c8f tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] validate_networks() for [('daf58feb-9527-44f2-90a3-a19a525b593d', None, None, None, None, None)] {{(pid=120864) validate_networks /opt/stack/nova/nova/network/neutron.py:2626}} Aug 27 02:31:11.505073 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.virt.hardware [None req-1ccd2ce9-ea66-4ba2-b616-dd2324a16c8f tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Flavor limits 0:0:0 {{(pid=120864) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Aug 27 02:31:11.505530 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.virt.hardware [None req-1ccd2ce9-ea66-4ba2-b616-dd2324a16c8f tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Image limits 0:0:0 {{(pid=120864) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Aug 27 02:31:11.506000 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.virt.hardware [None req-1ccd2ce9-ea66-4ba2-b616-dd2324a16c8f tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Flavor pref 0:0:0 {{(pid=120864) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Aug 27 02:31:11.506423 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.virt.hardware [None req-1ccd2ce9-ea66-4ba2-b616-dd2324a16c8f tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Image pref 0:0:0 {{(pid=120864) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Aug 27 02:31:11.506828 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.virt.hardware [None req-1ccd2ce9-ea66-4ba2-b616-dd2324a16c8f tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=120864) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Aug 27 02:31:11.516110 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-1ccd2ce9-ea66-4ba2-b616-dd2324a16c8f tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:31:11.516586 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-1ccd2ce9-ea66-4ba2-b616-dd2324a16c8f tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:31:11.517013 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-1ccd2ce9-ea66-4ba2-b616-dd2324a16c8f tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:31:11.517767 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-1ccd2ce9-ea66-4ba2-b616-dd2324a16c8f tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:31:11.518219 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-1ccd2ce9-ea66-4ba2-b616-dd2324a16c8f tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:31:11.518637 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-1ccd2ce9-ea66-4ba2-b616-dd2324a16c8f tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:31:11.536443 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.quota [None req-1ccd2ce9-ea66-4ba2-b616-dd2324a16c8f tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Getting quotas for project c938ae5fd22a43e8a01859da2a336b53. Resources: {'cores', 'ram', 'instances'} {{(pid=120864) _get_quotas /opt/stack/nova/nova/quota.py:393}} Aug 27 02:31:11.540798 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.quota [None req-1ccd2ce9-ea66-4ba2-b616-dd2324a16c8f tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Getting quotas for user d750e6e172d34d4a9333e0c059c5442b and project c938ae5fd22a43e8a01859da2a336b53. Resources: {'cores', 'ram', 'instances'} {{(pid=120864) _get_quotas /opt/stack/nova/nova/quota.py:383}} Aug 27 02:31:11.552134 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-1ccd2ce9-ea66-4ba2-b616-dd2324a16c8f tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Going to run 1 instances... {{(pid=120864) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Aug 27 02:31:11.554470 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-1ccd2ce9-ea66-4ba2-b616-dd2324a16c8f tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] [instance: fff5c233-ad7f-4022-8582-e8620f32ebd5] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='9fc30b36-b827-46f9-892d-30e808bc2b8a',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=120864) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Aug 27 02:31:11.606660 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-1ccd2ce9-ea66-4ba2-b616-dd2324a16c8f tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] 10.222.0.52 "POST /compute/v2.1/servers" status: 202 len: 402 microversion: 2.20 time: 0.881070 Aug 27 02:31:11.607702 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 417/828] 10.222.0.52 () {62 vars in 1188 bytes} [Sat Aug 27 02:31:10 2022] POST /compute/v2.1/servers => generated 402 bytes in 883 msecs (HTTP/1.1 202) 10 headers in 453 bytes (1 switches on core 0) Aug 27 02:31:11.619190 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-87f27832-31a6-4d88-88ff-04bfea502452 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:31:11.620077 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-87f27832-31a6-4d88-88ff-04bfea502452 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] [instance: fff5c233-ad7f-4022-8582-e8620f32ebd5] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:31:11.678264 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-87f27832-31a6-4d88-88ff-04bfea502452 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:31:11.686951 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-87f27832-31a6-4d88-88ff-04bfea502452 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:31:11.688485 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-87f27832-31a6-4d88-88ff-04bfea502452 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:31:11.689634 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-87f27832-31a6-4d88-88ff-04bfea502452 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] 10.222.0.52 "GET /compute/v2.1/servers/fff5c233-ad7f-4022-8582-e8620f32ebd5" status: 200 len: 1282 microversion: 2.20 time: 0.075276 Aug 27 02:31:11.690173 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 412/829] 10.222.0.52 () {60 vars in 1277 bytes} [Sat Aug 27 02:31:11 2022] GET /compute/v2.1/servers/fff5c233-ad7f-4022-8582-e8620f32ebd5 => generated 1282 bytes in 76 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 27 02:31:12.712518 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-9f712cb8-7b3f-488b-a810-af3d05250c59 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Calling method '>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:31:12.713418 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-9f712cb8-7b3f-488b-a810-af3d05250c59 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] [instance: fff5c233-ad7f-4022-8582-e8620f32ebd5] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:31:12.720199 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-9f712cb8-7b3f-488b-a810-af3d05250c59 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:31:12.720709 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-9f712cb8-7b3f-488b-a810-af3d05250c59 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:31:12.721173 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-9f712cb8-7b3f-488b-a810-af3d05250c59 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:31:12.721901 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-9f712cb8-7b3f-488b-a810-af3d05250c59 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:31:12.722353 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-9f712cb8-7b3f-488b-a810-af3d05250c59 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:31:12.722774 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-9f712cb8-7b3f-488b-a810-af3d05250c59 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:31:12.788088 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-9f712cb8-7b3f-488b-a810-af3d05250c59 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:31:12.796991 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-9f712cb8-7b3f-488b-a810-af3d05250c59 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:31:12.798536 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-9f712cb8-7b3f-488b-a810-af3d05250c59 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:31:12.800586 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-9f712cb8-7b3f-488b-a810-af3d05250c59 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] 10.222.0.52 "GET /compute/v2.1/servers/fff5c233-ad7f-4022-8582-e8620f32ebd5" status: 200 len: 1334 microversion: 2.20 time: 0.092093 Aug 27 02:31:12.801800 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 418/830] 10.222.0.52 () {60 vars in 1277 bytes} [Sat Aug 27 02:31:12 2022] GET /compute/v2.1/servers/fff5c233-ad7f-4022-8582-e8620f32ebd5 => generated 1334 bytes in 94 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 27 02:31:13.824596 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-29cca808-e95a-4b82-ac7b-30617c67096b tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:31:13.825231 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-29cca808-e95a-4b82-ac7b-30617c67096b tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] [instance: fff5c233-ad7f-4022-8582-e8620f32ebd5] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:31:13.831476 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-29cca808-e95a-4b82-ac7b-30617c67096b tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:31:13.831959 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-29cca808-e95a-4b82-ac7b-30617c67096b tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:31:13.832394 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-29cca808-e95a-4b82-ac7b-30617c67096b tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:31:13.833083 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-29cca808-e95a-4b82-ac7b-30617c67096b tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:31:13.833673 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-29cca808-e95a-4b82-ac7b-30617c67096b tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:31:13.834097 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-29cca808-e95a-4b82-ac7b-30617c67096b tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:31:14.080873 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-29cca808-e95a-4b82-ac7b-30617c67096b tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:31:14.089219 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-29cca808-e95a-4b82-ac7b-30617c67096b tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:31:14.091110 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-29cca808-e95a-4b82-ac7b-30617c67096b tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:31:14.092227 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-29cca808-e95a-4b82-ac7b-30617c67096b tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] 10.222.0.52 "GET /compute/v2.1/servers/fff5c233-ad7f-4022-8582-e8620f32ebd5" status: 200 len: 1408 microversion: 2.20 time: 0.272045 Aug 27 02:31:14.092809 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 413/831] 10.222.0.52 () {60 vars in 1277 bytes} [Sat Aug 27 02:31:13 2022] GET /compute/v2.1/servers/fff5c233-ad7f-4022-8582-e8620f32ebd5 => generated 1408 bytes in 273 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 27 02:31:14.408622 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [req-82e70c22-2aec-46fd-9fb6-8fcdf2eac3ea req-46063980-7857-4629-95a5-23708153c9c5 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "fff5c233-ad7f-4022-8582-e8620f32ebd5", "tag": "67fc5822-1682-4324-ae1c-b98d8d2d09e8"}]} {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 27 02:31:14.420611 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [req-82e70c22-2aec-46fd-9fb6-8fcdf2eac3ea req-46063980-7857-4629-95a5-23708153c9c5 service nova] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:31:14.421462 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [req-82e70c22-2aec-46fd-9fb6-8fcdf2eac3ea req-46063980-7857-4629-95a5-23708153c9c5 service nova] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:31:14.422136 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [req-82e70c22-2aec-46fd-9fb6-8fcdf2eac3ea req-46063980-7857-4629-95a5-23708153c9c5 service nova] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:31:14.434395 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.compute.server_external_events [req-82e70c22-2aec-46fd-9fb6-8fcdf2eac3ea req-46063980-7857-4629-95a5-23708153c9c5 service nova] Creating event network-changed:67fc5822-1682-4324-ae1c-b98d8d2d09e8 for instance fff5c233-ad7f-4022-8582-e8620f32ebd5 on hv-cinder-84630 Aug 27 02:31:14.439735 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [req-82e70c22-2aec-46fd-9fb6-8fcdf2eac3ea req-46063980-7857-4629-95a5-23708153c9c5 service nova] 10.222.0.52 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.035196 Aug 27 02:31:14.440342 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 419/832] 10.222.0.52 () {64 vars in 1317 bytes} [Sat Aug 27 02:31:14 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 37 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 27 02:31:15.117005 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-47ce685f-de8f-4bad-9ace-f0319a59d43d tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:31:15.117825 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-47ce685f-de8f-4bad-9ace-f0319a59d43d tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] [instance: fff5c233-ad7f-4022-8582-e8620f32ebd5] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:31:15.127704 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-47ce685f-de8f-4bad-9ace-f0319a59d43d tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:31:15.128546 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-47ce685f-de8f-4bad-9ace-f0319a59d43d tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:31:15.129298 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-47ce685f-de8f-4bad-9ace-f0319a59d43d tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:31:15.130429 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-47ce685f-de8f-4bad-9ace-f0319a59d43d tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:31:15.131174 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-47ce685f-de8f-4bad-9ace-f0319a59d43d tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:31:15.131866 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-47ce685f-de8f-4bad-9ace-f0319a59d43d tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:31:15.264481 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-47ce685f-de8f-4bad-9ace-f0319a59d43d tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:31:15.272776 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-47ce685f-de8f-4bad-9ace-f0319a59d43d tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:31:15.274253 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-47ce685f-de8f-4bad-9ace-f0319a59d43d tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:31:15.275352 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-47ce685f-de8f-4bad-9ace-f0319a59d43d tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] 10.222.0.52 "GET /compute/v2.1/servers/fff5c233-ad7f-4022-8582-e8620f32ebd5" status: 200 len: 1408 microversion: 2.20 time: 0.163178 Aug 27 02:31:15.276008 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 414/833] 10.222.0.52 () {60 vars in 1277 bytes} [Sat Aug 27 02:31:15 2022] GET /compute/v2.1/servers/fff5c233-ad7f-4022-8582-e8620f32ebd5 => generated 1408 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 27 02:31:16.299071 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-21cfbea0-b36b-4f73-8bb5-a7403ef1f771 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Calling method '>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:31:16.299753 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-21cfbea0-b36b-4f73-8bb5-a7403ef1f771 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] [instance: fff5c233-ad7f-4022-8582-e8620f32ebd5] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:31:16.306764 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-21cfbea0-b36b-4f73-8bb5-a7403ef1f771 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:31:16.307494 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-21cfbea0-b36b-4f73-8bb5-a7403ef1f771 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:31:16.307694 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-21cfbea0-b36b-4f73-8bb5-a7403ef1f771 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:31:16.308397 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-21cfbea0-b36b-4f73-8bb5-a7403ef1f771 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:31:16.308866 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-21cfbea0-b36b-4f73-8bb5-a7403ef1f771 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:31:16.309347 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-21cfbea0-b36b-4f73-8bb5-a7403ef1f771 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:31:16.455939 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-21cfbea0-b36b-4f73-8bb5-a7403ef1f771 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:31:16.467740 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-21cfbea0-b36b-4f73-8bb5-a7403ef1f771 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:31:16.469859 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-21cfbea0-b36b-4f73-8bb5-a7403ef1f771 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:31:16.471433 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-21cfbea0-b36b-4f73-8bb5-a7403ef1f771 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] 10.222.0.52 "GET /compute/v2.1/servers/fff5c233-ad7f-4022-8582-e8620f32ebd5" status: 200 len: 1408 microversion: 2.20 time: 0.176917 Aug 27 02:31:16.472423 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 420/834] 10.222.0.52 () {60 vars in 1277 bytes} [Sat Aug 27 02:31:16 2022] GET /compute/v2.1/servers/fff5c233-ad7f-4022-8582-e8620f32ebd5 => generated 1408 bytes in 179 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 27 02:31:17.496444 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-1b7c5e2e-fa6a-4bcb-a3fd-fb3072bd9faf tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:31:17.497402 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-1b7c5e2e-fa6a-4bcb-a3fd-fb3072bd9faf tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] [instance: fff5c233-ad7f-4022-8582-e8620f32ebd5] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:31:17.505382 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-1b7c5e2e-fa6a-4bcb-a3fd-fb3072bd9faf tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:31:17.505865 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-1b7c5e2e-fa6a-4bcb-a3fd-fb3072bd9faf tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:31:17.506302 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-1b7c5e2e-fa6a-4bcb-a3fd-fb3072bd9faf tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:31:17.507019 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-1b7c5e2e-fa6a-4bcb-a3fd-fb3072bd9faf tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:31:17.507471 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-1b7c5e2e-fa6a-4bcb-a3fd-fb3072bd9faf tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:31:17.507894 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-1b7c5e2e-fa6a-4bcb-a3fd-fb3072bd9faf tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:31:17.657727 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-1b7c5e2e-fa6a-4bcb-a3fd-fb3072bd9faf tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:31:17.666377 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-1b7c5e2e-fa6a-4bcb-a3fd-fb3072bd9faf tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:31:17.668102 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-1b7c5e2e-fa6a-4bcb-a3fd-fb3072bd9faf tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:31:17.671808 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-1b7c5e2e-fa6a-4bcb-a3fd-fb3072bd9faf tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] 10.222.0.52 "GET /compute/v2.1/servers/fff5c233-ad7f-4022-8582-e8620f32ebd5" status: 200 len: 1408 microversion: 2.20 time: 0.176990 Aug 27 02:31:17.671808 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 415/835] 10.222.0.52 () {60 vars in 1277 bytes} [Sat Aug 27 02:31:17 2022] GET /compute/v2.1/servers/fff5c233-ad7f-4022-8582-e8620f32ebd5 => generated 1408 bytes in 178 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 27 02:31:18.692406 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-67116c0b-f536-4029-856d-e6854f7233e1 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Calling method '>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:31:18.693048 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-67116c0b-f536-4029-856d-e6854f7233e1 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] [instance: fff5c233-ad7f-4022-8582-e8620f32ebd5] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:31:18.700174 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-67116c0b-f536-4029-856d-e6854f7233e1 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:31:18.700680 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-67116c0b-f536-4029-856d-e6854f7233e1 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:31:18.701111 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-67116c0b-f536-4029-856d-e6854f7233e1 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:31:18.701846 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-67116c0b-f536-4029-856d-e6854f7233e1 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:31:18.702262 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-67116c0b-f536-4029-856d-e6854f7233e1 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:31:18.702677 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-67116c0b-f536-4029-856d-e6854f7233e1 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:31:18.848094 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-67116c0b-f536-4029-856d-e6854f7233e1 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:31:18.856731 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-67116c0b-f536-4029-856d-e6854f7233e1 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:31:18.858185 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-67116c0b-f536-4029-856d-e6854f7233e1 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:31:18.859289 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-67116c0b-f536-4029-856d-e6854f7233e1 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] 10.222.0.52 "GET /compute/v2.1/servers/fff5c233-ad7f-4022-8582-e8620f32ebd5" status: 200 len: 1408 microversion: 2.20 time: 0.171193 Aug 27 02:31:18.859875 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 421/836] 10.222.0.52 () {60 vars in 1277 bytes} [Sat Aug 27 02:31:18 2022] GET /compute/v2.1/servers/fff5c233-ad7f-4022-8582-e8620f32ebd5 => generated 1408 bytes in 172 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 27 02:31:19.881796 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-73f97323-a885-48ec-b2fd-266e8f91120a tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:31:19.882440 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-73f97323-a885-48ec-b2fd-266e8f91120a tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] [instance: fff5c233-ad7f-4022-8582-e8620f32ebd5] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:31:19.889271 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-73f97323-a885-48ec-b2fd-266e8f91120a tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:31:19.889769 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-73f97323-a885-48ec-b2fd-266e8f91120a tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:31:19.890199 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-73f97323-a885-48ec-b2fd-266e8f91120a tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:31:19.890902 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-73f97323-a885-48ec-b2fd-266e8f91120a tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:31:19.891349 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-73f97323-a885-48ec-b2fd-266e8f91120a tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:31:19.891769 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-73f97323-a885-48ec-b2fd-266e8f91120a tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:31:20.031875 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-73f97323-a885-48ec-b2fd-266e8f91120a tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:31:20.041522 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-73f97323-a885-48ec-b2fd-266e8f91120a tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:31:20.042969 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-73f97323-a885-48ec-b2fd-266e8f91120a tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:31:20.044089 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-73f97323-a885-48ec-b2fd-266e8f91120a tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] 10.222.0.52 "GET /compute/v2.1/servers/fff5c233-ad7f-4022-8582-e8620f32ebd5" status: 200 len: 1408 microversion: 2.20 time: 0.166281 Aug 27 02:31:20.044635 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 416/837] 10.222.0.52 () {60 vars in 1277 bytes} [Sat Aug 27 02:31:19 2022] GET /compute/v2.1/servers/fff5c233-ad7f-4022-8582-e8620f32ebd5 => generated 1408 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 27 02:31:20.491664 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [req-62e497a6-73ca-497b-a8d7-515513e2647c req-f86f4bca-a42f-4cb3-86ab-ccad49fa8b05 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "fff5c233-ad7f-4022-8582-e8620f32ebd5", "name": "network-vif-plugged", "status": "completed", "tag": "67fc5822-1682-4324-ae1c-b98d8d2d09e8"}]} {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 27 02:31:20.502201 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [req-62e497a6-73ca-497b-a8d7-515513e2647c req-f86f4bca-a42f-4cb3-86ab-ccad49fa8b05 service nova] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:31:20.502606 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [req-62e497a6-73ca-497b-a8d7-515513e2647c req-f86f4bca-a42f-4cb3-86ab-ccad49fa8b05 service nova] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:31:20.503031 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [req-62e497a6-73ca-497b-a8d7-515513e2647c req-f86f4bca-a42f-4cb3-86ab-ccad49fa8b05 service nova] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:31:20.517435 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.compute.server_external_events [req-62e497a6-73ca-497b-a8d7-515513e2647c req-f86f4bca-a42f-4cb3-86ab-ccad49fa8b05 service nova] Creating event network-vif-plugged:67fc5822-1682-4324-ae1c-b98d8d2d09e8 for instance fff5c233-ad7f-4022-8582-e8620f32ebd5 on hv-cinder-84630 Aug 27 02:31:20.522974 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [req-62e497a6-73ca-497b-a8d7-515513e2647c req-f86f4bca-a42f-4cb3-86ab-ccad49fa8b05 service nova] 10.222.0.52 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.035082 Aug 27 02:31:20.523588 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 422/838] 10.222.0.52 () {64 vars in 1317 bytes} [Sat Aug 27 02:31:20 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 37 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 27 02:31:21.066978 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-d0bd341d-7bbd-4907-9848-ca7f80ca427b tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:31:21.067608 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-d0bd341d-7bbd-4907-9848-ca7f80ca427b tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] [instance: fff5c233-ad7f-4022-8582-e8620f32ebd5] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:31:21.074550 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-d0bd341d-7bbd-4907-9848-ca7f80ca427b tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:31:21.075037 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-d0bd341d-7bbd-4907-9848-ca7f80ca427b tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:31:21.075471 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-d0bd341d-7bbd-4907-9848-ca7f80ca427b tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:31:21.076165 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-d0bd341d-7bbd-4907-9848-ca7f80ca427b tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:31:21.076613 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-d0bd341d-7bbd-4907-9848-ca7f80ca427b tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:31:21.077025 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-d0bd341d-7bbd-4907-9848-ca7f80ca427b tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:31:21.215097 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-d0bd341d-7bbd-4907-9848-ca7f80ca427b tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:31:21.223531 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-d0bd341d-7bbd-4907-9848-ca7f80ca427b tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:31:21.224987 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-d0bd341d-7bbd-4907-9848-ca7f80ca427b tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:31:21.226137 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-d0bd341d-7bbd-4907-9848-ca7f80ca427b tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] 10.222.0.52 "GET /compute/v2.1/servers/fff5c233-ad7f-4022-8582-e8620f32ebd5" status: 200 len: 1408 microversion: 2.20 time: 0.163194 Aug 27 02:31:21.226717 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 417/839] 10.222.0.52 () {60 vars in 1277 bytes} [Sat Aug 27 02:31:21 2022] GET /compute/v2.1/servers/fff5c233-ad7f-4022-8582-e8620f32ebd5 => generated 1408 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 27 02:31:22.247906 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG dbcounter [-] [120864] Writing DB stats nova_cell0:SELECT=2 {{(pid=120864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 27 02:31:22.248595 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG dbcounter [-] [120864] Writing DB stats nova_cell0:SELECT=6 {{(pid=120864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 27 02:31:22.252734 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-7b1fbf16-a57c-4b04-a442-a2ca5ca93176 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Calling method '>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:31:22.253561 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-7b1fbf16-a57c-4b04-a442-a2ca5ca93176 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] [instance: fff5c233-ad7f-4022-8582-e8620f32ebd5] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:31:22.260960 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-7b1fbf16-a57c-4b04-a442-a2ca5ca93176 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:31:22.261468 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-7b1fbf16-a57c-4b04-a442-a2ca5ca93176 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:31:22.261891 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-7b1fbf16-a57c-4b04-a442-a2ca5ca93176 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:31:22.262570 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-7b1fbf16-a57c-4b04-a442-a2ca5ca93176 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:31:22.263012 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-7b1fbf16-a57c-4b04-a442-a2ca5ca93176 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:31:22.263428 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-7b1fbf16-a57c-4b04-a442-a2ca5ca93176 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:31:22.404061 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-7b1fbf16-a57c-4b04-a442-a2ca5ca93176 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:31:22.412800 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-7b1fbf16-a57c-4b04-a442-a2ca5ca93176 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:31:22.414276 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-7b1fbf16-a57c-4b04-a442-a2ca5ca93176 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:31:22.415423 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-7b1fbf16-a57c-4b04-a442-a2ca5ca93176 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] 10.222.0.52 "GET /compute/v2.1/servers/fff5c233-ad7f-4022-8582-e8620f32ebd5" status: 200 len: 1408 microversion: 2.20 time: 0.169087 Aug 27 02:31:22.415992 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 423/840] 10.222.0.52 () {60 vars in 1277 bytes} [Sat Aug 27 02:31:22 2022] GET /compute/v2.1/servers/fff5c233-ad7f-4022-8582-e8620f32ebd5 => generated 1408 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 27 02:31:23.438034 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-e017c23e-d5fd-42f6-b7fb-eee86c1e147b tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:31:23.438731 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-e017c23e-d5fd-42f6-b7fb-eee86c1e147b tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] [instance: fff5c233-ad7f-4022-8582-e8620f32ebd5] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:31:23.440536 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG dbcounter [-] [120863] Writing DB stats nova_cell0:SELECT=6 {{(pid=120863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 27 02:31:23.447217 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-e017c23e-d5fd-42f6-b7fb-eee86c1e147b tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:31:23.447807 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-e017c23e-d5fd-42f6-b7fb-eee86c1e147b tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:31:23.448238 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-e017c23e-d5fd-42f6-b7fb-eee86c1e147b tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:31:23.448930 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-e017c23e-d5fd-42f6-b7fb-eee86c1e147b tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:31:23.449402 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-e017c23e-d5fd-42f6-b7fb-eee86c1e147b tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:31:23.449819 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-e017c23e-d5fd-42f6-b7fb-eee86c1e147b tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:31:23.585860 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-e017c23e-d5fd-42f6-b7fb-eee86c1e147b tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:31:23.598473 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-e017c23e-d5fd-42f6-b7fb-eee86c1e147b tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:31:23.600754 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-e017c23e-d5fd-42f6-b7fb-eee86c1e147b tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:31:23.602644 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-e017c23e-d5fd-42f6-b7fb-eee86c1e147b tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] 10.222.0.52 "GET /compute/v2.1/servers/fff5c233-ad7f-4022-8582-e8620f32ebd5" status: 200 len: 1601 microversion: 2.20 time: 0.168342 Aug 27 02:31:23.603507 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 418/841] 10.222.0.52 () {60 vars in 1277 bytes} [Sat Aug 27 02:31:23 2022] GET /compute/v2.1/servers/fff5c233-ad7f-4022-8582-e8620f32ebd5 => generated 1601 bytes in 170 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Aug 27 02:31:23.631153 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-331e2b55-926e-4e12-a73c-ea78234e6e97 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Calling method '>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:31:23.631766 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-331e2b55-926e-4e12-a73c-ea78234e6e97 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] [instance: fff5c233-ad7f-4022-8582-e8620f32ebd5] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:31:23.638478 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-331e2b55-926e-4e12-a73c-ea78234e6e97 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:31:23.638962 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-331e2b55-926e-4e12-a73c-ea78234e6e97 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:31:23.639395 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-331e2b55-926e-4e12-a73c-ea78234e6e97 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:31:23.640090 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-331e2b55-926e-4e12-a73c-ea78234e6e97 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:31:23.640537 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-331e2b55-926e-4e12-a73c-ea78234e6e97 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:31:23.640959 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-331e2b55-926e-4e12-a73c-ea78234e6e97 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:31:23.726276 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-331e2b55-926e-4e12-a73c-ea78234e6e97 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] 10.222.0.52 "GET /compute/v2.1/servers/fff5c233-ad7f-4022-8582-e8620f32ebd5/os-interface" status: 200 len: 285 microversion: 2.20 time: 0.099009 Aug 27 02:31:23.727004 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 424/842] 10.222.0.52 () {60 vars in 1316 bytes} [Sat Aug 27 02:31:23 2022] GET /compute/v2.1/servers/fff5c233-ad7f-4022-8582-e8620f32ebd5/os-interface => generated 285 bytes in 100 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Aug 27 02:31:24.723010 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [req-a470fa63-915d-455d-ac06-97dbdbcd9262 req-9ffb540d-9918-42cb-b239-f7875d1251d1 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "fff5c233-ad7f-4022-8582-e8620f32ebd5", "tag": "67fc5822-1682-4324-ae1c-b98d8d2d09e8"}]} {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 27 02:31:24.733601 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [req-a470fa63-915d-455d-ac06-97dbdbcd9262 req-9ffb540d-9918-42cb-b239-f7875d1251d1 service nova] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:31:24.734090 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [req-a470fa63-915d-455d-ac06-97dbdbcd9262 req-9ffb540d-9918-42cb-b239-f7875d1251d1 service nova] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:31:24.734513 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [req-a470fa63-915d-455d-ac06-97dbdbcd9262 req-9ffb540d-9918-42cb-b239-f7875d1251d1 service nova] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:31:24.749228 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.compute.server_external_events [req-a470fa63-915d-455d-ac06-97dbdbcd9262 req-9ffb540d-9918-42cb-b239-f7875d1251d1 service nova] Creating event network-changed:67fc5822-1682-4324-ae1c-b98d8d2d09e8 for instance fff5c233-ad7f-4022-8582-e8620f32ebd5 on hv-cinder-84630 Aug 27 02:31:24.754511 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [req-a470fa63-915d-455d-ac06-97dbdbcd9262 req-9ffb540d-9918-42cb-b239-f7875d1251d1 service nova] 10.222.0.52 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.035117 Aug 27 02:31:24.755332 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 419/843] 10.222.0.52 () {64 vars in 1317 bytes} [Sat Aug 27 02:31:24 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 37 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 27 02:31:27.355049 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-2dbb79a9-d522-4072-9b27-af7cf5758eb5 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Calling method '>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:31:27.355622 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-2dbb79a9-d522-4072-9b27-af7cf5758eb5 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] [instance: 98e4cfeb-d49e-4888-a2c0-40a9e8af90d3] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:31:27.362655 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-2dbb79a9-d522-4072-9b27-af7cf5758eb5 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:31:27.363207 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-2dbb79a9-d522-4072-9b27-af7cf5758eb5 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:31:27.363633 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-2dbb79a9-d522-4072-9b27-af7cf5758eb5 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:31:27.364320 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-2dbb79a9-d522-4072-9b27-af7cf5758eb5 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:31:27.364941 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-2dbb79a9-d522-4072-9b27-af7cf5758eb5 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:31:27.365386 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-2dbb79a9-d522-4072-9b27-af7cf5758eb5 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:31:27.449292 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.objects.instance [None req-2dbb79a9-d522-4072-9b27-af7cf5758eb5 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lazy-loading 'services' on Instance uuid 98e4cfeb-d49e-4888-a2c0-40a9e8af90d3 {{(pid=120864) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Aug 27 02:31:27.523125 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-2dbb79a9-d522-4072-9b27-af7cf5758eb5 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] 10.222.0.52 "DELETE /compute/v2.1/servers/98e4cfeb-d49e-4888-a2c0-40a9e8af90d3/os-volume_attachments/2a8726a0-0c8d-4001-aaca-d11c877d0d88" status: 202 len: 0 microversion: 2.1 time: 0.172668 Aug 27 02:31:27.523743 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 425/844] 10.222.0.52 () {58 vars in 1416 bytes} [Sat Aug 27 02:31:27 2022] DELETE /compute/v2.1/servers/98e4cfeb-d49e-4888-a2c0-40a9e8af90d3/os-volume_attachments/2a8726a0-0c8d-4001-aaca-d11c877d0d88 => generated 0 bytes in 174 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Aug 27 02:31:27.533831 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-f5093503-fa63-454e-ab12-d2d49b10038a tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:31:27.536063 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-f5093503-fa63-454e-ab12-d2d49b10038a tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] [instance: 98e4cfeb-d49e-4888-a2c0-40a9e8af90d3] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:31:27.546294 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-f5093503-fa63-454e-ab12-d2d49b10038a tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:31:27.548722 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-f5093503-fa63-454e-ab12-d2d49b10038a tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:31:27.549078 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-f5093503-fa63-454e-ab12-d2d49b10038a tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:31:27.550052 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-f5093503-fa63-454e-ab12-d2d49b10038a tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:31:27.550507 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-f5093503-fa63-454e-ab12-d2d49b10038a tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:31:27.551055 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-f5093503-fa63-454e-ab12-d2d49b10038a tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:31:27.584822 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-f5093503-fa63-454e-ab12-d2d49b10038a tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] 10.222.0.52 "GET /compute/v2.1/servers/98e4cfeb-d49e-4888-a2c0-40a9e8af90d3/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.055367 Aug 27 02:31:27.585749 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 420/845] 10.222.0.52 () {58 vars in 1302 bytes} [Sat Aug 27 02:31:27 2022] GET /compute/v2.1/servers/98e4cfeb-d49e-4888-a2c0-40a9e8af90d3/os-volume_attachments => generated 197 bytes in 58 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 27 02:31:28.600430 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-97e28e39-84a2-4951-90e0-867af78f7d49 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Calling method '>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:31:28.602856 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-97e28e39-84a2-4951-90e0-867af78f7d49 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] [instance: 98e4cfeb-d49e-4888-a2c0-40a9e8af90d3] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:31:28.611005 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-97e28e39-84a2-4951-90e0-867af78f7d49 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:31:28.611566 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-97e28e39-84a2-4951-90e0-867af78f7d49 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:31:28.611975 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-97e28e39-84a2-4951-90e0-867af78f7d49 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:31:28.612696 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-97e28e39-84a2-4951-90e0-867af78f7d49 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:31:28.613110 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-97e28e39-84a2-4951-90e0-867af78f7d49 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:31:28.613696 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-97e28e39-84a2-4951-90e0-867af78f7d49 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:31:28.646577 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-97e28e39-84a2-4951-90e0-867af78f7d49 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] 10.222.0.52 "GET /compute/v2.1/servers/98e4cfeb-d49e-4888-a2c0-40a9e8af90d3/os-volume_attachments" status: 200 len: 25 microversion: 2.1 time: 0.051212 Aug 27 02:31:28.647141 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 426/846] 10.222.0.52 () {58 vars in 1302 bytes} [Sat Aug 27 02:31:28 2022] GET /compute/v2.1/servers/98e4cfeb-d49e-4888-a2c0-40a9e8af90d3/os-volume_attachments => generated 25 bytes in 53 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Aug 27 02:31:28.743252 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-9d86de98-21b7-4c23-b810-0eef08480ba7 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:31:28.743821 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-9d86de98-21b7-4c23-b810-0eef08480ba7 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] [instance: 98e4cfeb-d49e-4888-a2c0-40a9e8af90d3] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:31:28.751373 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-9d86de98-21b7-4c23-b810-0eef08480ba7 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:31:28.751877 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-9d86de98-21b7-4c23-b810-0eef08480ba7 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:31:28.752309 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-9d86de98-21b7-4c23-b810-0eef08480ba7 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:31:28.752993 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-9d86de98-21b7-4c23-b810-0eef08480ba7 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:31:28.753470 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-9d86de98-21b7-4c23-b810-0eef08480ba7 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:31:28.754004 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-9d86de98-21b7-4c23-b810-0eef08480ba7 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:31:28.781265 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-9d86de98-21b7-4c23-b810-0eef08480ba7 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] [instance: 98e4cfeb-d49e-4888-a2c0-40a9e8af90d3] Going to try to terminate instance {{(pid=120863) delete /opt/stack/nova/nova/compute/api.py:2719}} Aug 27 02:31:28.830118 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG dbcounter [-] [120863] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=120863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 27 02:31:28.859233 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-9d86de98-21b7-4c23-b810-0eef08480ba7 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] 10.222.0.52 "DELETE /compute/v2.1/servers/98e4cfeb-d49e-4888-a2c0-40a9e8af90d3" status: 204 len: 0 microversion: 2.1 time: 0.120066 Aug 27 02:31:28.859735 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 421/847] 10.222.0.52 () {58 vars in 1239 bytes} [Sat Aug 27 02:31:28 2022] DELETE /compute/v2.1/servers/98e4cfeb-d49e-4888-a2c0-40a9e8af90d3 => generated 0 bytes in 122 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Aug 27 02:31:28.868719 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-172fb248-0d64-4af6-87bb-a2481ecb76f3 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Calling method '>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:31:28.869368 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-172fb248-0d64-4af6-87bb-a2481ecb76f3 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] [instance: 98e4cfeb-d49e-4888-a2c0-40a9e8af90d3] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:31:28.875650 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-172fb248-0d64-4af6-87bb-a2481ecb76f3 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:31:28.876134 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-172fb248-0d64-4af6-87bb-a2481ecb76f3 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:31:28.876568 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-172fb248-0d64-4af6-87bb-a2481ecb76f3 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:31:28.877286 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-172fb248-0d64-4af6-87bb-a2481ecb76f3 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:31:28.877738 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-172fb248-0d64-4af6-87bb-a2481ecb76f3 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:31:28.878280 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-172fb248-0d64-4af6-87bb-a2481ecb76f3 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:31:28.882487 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG dbcounter [-] [120864] Writing DB stats nova_cell1:SELECT=96,nova_cell1:INSERT=3,nova_cell1:UPDATE=1 {{(pid=120864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 27 02:31:29.017014 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-172fb248-0d64-4af6-87bb-a2481ecb76f3 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd945d512ae034ec08838d0a3afde2429', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '880ac4003248441c809a76cfd1b221eb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:31:29.027496 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-172fb248-0d64-4af6-87bb-a2481ecb76f3 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] 10.222.0.52 "GET /compute/v2.1/servers/98e4cfeb-d49e-4888-a2c0-40a9e8af90d3" status: 200 len: 1689 microversion: 2.1 time: 0.162656 Aug 27 02:31:29.028071 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 427/848] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:31:28 2022] GET /compute/v2.1/servers/98e4cfeb-d49e-4888-a2c0-40a9e8af90d3 => generated 1689 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:31:30.053581 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-6136568c-21d5-429b-8b8d-18de3b611891 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:31:30.054169 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-6136568c-21d5-429b-8b8d-18de3b611891 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] [instance: 98e4cfeb-d49e-4888-a2c0-40a9e8af90d3] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:31:30.056252 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG dbcounter [-] [120863] Writing DB stats nova_api:SELECT=55,nova_api:INSERT=1,nova_api:UPDATE=1 {{(pid=120863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 27 02:31:30.062119 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-6136568c-21d5-429b-8b8d-18de3b611891 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:31:30.062603 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-6136568c-21d5-429b-8b8d-18de3b611891 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:31:30.063044 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-6136568c-21d5-429b-8b8d-18de3b611891 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:31:30.063741 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-6136568c-21d5-429b-8b8d-18de3b611891 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:31:30.064189 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-6136568c-21d5-429b-8b8d-18de3b611891 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:31:30.064685 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-6136568c-21d5-429b-8b8d-18de3b611891 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:31:30.206408 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-6136568c-21d5-429b-8b8d-18de3b611891 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd945d512ae034ec08838d0a3afde2429', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '880ac4003248441c809a76cfd1b221eb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:31:30.214442 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-6136568c-21d5-429b-8b8d-18de3b611891 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] 10.222.0.52 "GET /compute/v2.1/servers/98e4cfeb-d49e-4888-a2c0-40a9e8af90d3" status: 200 len: 1689 microversion: 2.1 time: 0.165206 Aug 27 02:31:30.215022 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 422/849] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:31:30 2022] GET /compute/v2.1/servers/98e4cfeb-d49e-4888-a2c0-40a9e8af90d3 => generated 1689 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:31:31.239436 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-67b63ab1-5bb6-443b-98f1-bdbe111276d6 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Calling method '>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:31:31.240342 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-67b63ab1-5bb6-443b-98f1-bdbe111276d6 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] [instance: 98e4cfeb-d49e-4888-a2c0-40a9e8af90d3] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:31:31.243842 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG dbcounter [-] [120864] Writing DB stats nova_api:SELECT=58,nova_api:UPDATE=1,nova_api:DELETE=1,nova_api:INSERT=3 {{(pid=120864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 27 02:31:31.249883 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-67b63ab1-5bb6-443b-98f1-bdbe111276d6 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:31:31.250373 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-67b63ab1-5bb6-443b-98f1-bdbe111276d6 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:31:31.250837 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-67b63ab1-5bb6-443b-98f1-bdbe111276d6 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:31:31.251534 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-67b63ab1-5bb6-443b-98f1-bdbe111276d6 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:31:31.251993 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-67b63ab1-5bb6-443b-98f1-bdbe111276d6 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:31:31.252404 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-67b63ab1-5bb6-443b-98f1-bdbe111276d6 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:31:31.392961 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-67b63ab1-5bb6-443b-98f1-bdbe111276d6 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd945d512ae034ec08838d0a3afde2429', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '880ac4003248441c809a76cfd1b221eb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:31:31.401641 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-67b63ab1-5bb6-443b-98f1-bdbe111276d6 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] 10.222.0.52 "GET /compute/v2.1/servers/98e4cfeb-d49e-4888-a2c0-40a9e8af90d3" status: 200 len: 1689 microversion: 2.1 time: 0.167964 Aug 27 02:31:31.402186 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 428/850] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:31:31 2022] GET /compute/v2.1/servers/98e4cfeb-d49e-4888-a2c0-40a9e8af90d3 => generated 1689 bytes in 170 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:31:32.423942 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-3eb75ec7-f25d-4eec-b732-a78a16f77d37 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:31:32.424540 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-3eb75ec7-f25d-4eec-b732-a78a16f77d37 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] [instance: 98e4cfeb-d49e-4888-a2c0-40a9e8af90d3] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:31:32.431823 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-3eb75ec7-f25d-4eec-b732-a78a16f77d37 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:31:32.432809 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-3eb75ec7-f25d-4eec-b732-a78a16f77d37 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:31:32.433520 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-3eb75ec7-f25d-4eec-b732-a78a16f77d37 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:31:32.434231 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-3eb75ec7-f25d-4eec-b732-a78a16f77d37 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:31:32.434691 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-3eb75ec7-f25d-4eec-b732-a78a16f77d37 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:31:32.435122 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-3eb75ec7-f25d-4eec-b732-a78a16f77d37 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:31:32.508880 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-3eb75ec7-f25d-4eec-b732-a78a16f77d37 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd945d512ae034ec08838d0a3afde2429', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '880ac4003248441c809a76cfd1b221eb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:31:32.517785 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-3eb75ec7-f25d-4eec-b732-a78a16f77d37 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] 10.222.0.52 "GET /compute/v2.1/servers/98e4cfeb-d49e-4888-a2c0-40a9e8af90d3" status: 200 len: 1621 microversion: 2.1 time: 0.097395 Aug 27 02:31:32.518424 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 423/851] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:31:32 2022] GET /compute/v2.1/servers/98e4cfeb-d49e-4888-a2c0-40a9e8af90d3 => generated 1621 bytes in 99 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 27 02:31:32.690371 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [req-5c618499-27a3-4ee9-b9e4-c16c438936a3 req-3bca4098-b705-44cd-a4a4-c3f989da8aa6 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "98e4cfeb-d49e-4888-a2c0-40a9e8af90d3", "name": "network-vif-deleted", "tag": "bf498aa0-7dfa-466b-b4f0-3a5d8f8703e2"}]} {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 27 02:31:32.709183 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [req-5c618499-27a3-4ee9-b9e4-c16c438936a3 req-3bca4098-b705-44cd-a4a4-c3f989da8aa6 service nova] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:31:32.709931 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [req-5c618499-27a3-4ee9-b9e4-c16c438936a3 req-3bca4098-b705-44cd-a4a4-c3f989da8aa6 service nova] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:31:32.710510 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [req-5c618499-27a3-4ee9-b9e4-c16c438936a3 req-3bca4098-b705-44cd-a4a4-c3f989da8aa6 service nova] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:31:32.726572 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.compute.server_external_events [req-5c618499-27a3-4ee9-b9e4-c16c438936a3 req-3bca4098-b705-44cd-a4a4-c3f989da8aa6 service nova] Creating event network-vif-deleted:bf498aa0-7dfa-466b-b4f0-3a5d8f8703e2 for instance 98e4cfeb-d49e-4888-a2c0-40a9e8af90d3 on hv-cinder-84630 Aug 27 02:31:32.731946 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [req-5c618499-27a3-4ee9-b9e4-c16c438936a3 req-3bca4098-b705-44cd-a4a4-c3f989da8aa6 service nova] 10.222.0.52 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.048424 Aug 27 02:31:32.735798 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 429/852] 10.222.0.52 () {64 vars in 1317 bytes} [Sat Aug 27 02:31:32 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 53 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Aug 27 02:31:33.540144 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-1b1bac7a-3e13-4455-bb72-6daca9410ae1 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:31:33.541452 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-1b1bac7a-3e13-4455-bb72-6daca9410ae1 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] [instance: 98e4cfeb-d49e-4888-a2c0-40a9e8af90d3] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:31:33.548774 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-1b1bac7a-3e13-4455-bb72-6daca9410ae1 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:31:33.549568 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-1b1bac7a-3e13-4455-bb72-6daca9410ae1 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:31:33.550012 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-1b1bac7a-3e13-4455-bb72-6daca9410ae1 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:31:33.550738 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-1b1bac7a-3e13-4455-bb72-6daca9410ae1 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:31:33.551199 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-1b1bac7a-3e13-4455-bb72-6daca9410ae1 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:31:33.551817 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-1b1bac7a-3e13-4455-bb72-6daca9410ae1 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:31:33.566244 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.wsgi [None req-1b1bac7a-3e13-4455-bb72-6daca9410ae1 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] HTTP exception thrown: Instance 98e4cfeb-d49e-4888-a2c0-40a9e8af90d3 could not be found. Aug 27 02:31:33.567866 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-1b1bac7a-3e13-4455-bb72-6daca9410ae1 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Request method failure captured: Aug 27 02:31:33.567866 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: request: GET /compute/v2.1/servers/98e4cfeb-d49e-4888-a2c0-40a9e8af90d3 HTTP/1.1 Aug 27 02:31:33.567866 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: Accept: application/json Aug 27 02:31:33.567866 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: Accept-Encoding: identity Aug 27 02:31:33.567866 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: Connection: close Aug 27 02:31:33.567866 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: Content-Length: 0 Aug 27 02:31:33.567866 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: Content-Type: application/json Aug 27 02:31:33.567866 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: Host: 10.222.0.52 Aug 27 02:31:33.567866 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: Openstack-System-Scope: None Aug 27 02:31:33.567866 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: User-Agent: python-urllib3/1.26.11 Aug 27 02:31:33.567866 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-Auth-Token: gAAAAABjCX_mTgf5582eaPxg7VeVEFNV70u35gVl4tYbLlqoLLSOTdT51sTqNUwxXgJqsRZlTyjbNrsB5eN_l1XXJng0l2G5rQDKOgGEffO457OUUMgEUK1evfhiyjkwW8ZvB7sHsdxG4zKRGU2G452QU2-KddZbqwFS3zX2uIy2nVW08GLJmCc Aug 27 02:31:33.567866 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-Domain-Id: None Aug 27 02:31:33.567866 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-Domain-Name: None Aug 27 02:31:33.567866 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-Identity-Status: Confirmed Aug 27 02:31:33.567866 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-Is-Admin-Project: True Aug 27 02:31:33.567866 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-Project-Domain-Id: default Aug 27 02:31:33.567866 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-Project-Domain-Name: Default Aug 27 02:31:33.567866 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-Project-Id: 880ac4003248441c809a76cfd1b221eb Aug 27 02:31:33.567866 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-Project-Name: tempest-ServerRescueNegativeTestJSON-1551042004 Aug 27 02:31:33.567866 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-Role: reader,member Aug 27 02:31:33.567866 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-Roles: reader,member Aug 27 02:31:33.569463 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-Service-Catalog: [{"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.52:8080", "publicURL": "http://10.222.0.52:8080/v1/AUTH_880ac4003248441c809a76cfd1b221eb"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52:9696/networking"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/compute/v2/880ac4003248441c809a76cfd1b221eb"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/image"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/volume/v3/880ac4003248441c809a76cfd1b221eb"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/identity"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/volume/v3/880ac4003248441c809a76cfd1b221eb"}]}] Aug 27 02:31:33.569463 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-Tenant: tempest-ServerRescueNegativeTestJSON-1551042004 Aug 27 02:31:33.569463 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-Tenant-Id: 880ac4003248441c809a76cfd1b221eb Aug 27 02:31:33.569463 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-Tenant-Name: tempest-ServerRescueNegativeTestJSON-1551042004 Aug 27 02:31:33.569463 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-User: tempest-ServerRescueNegativeTestJSON-1551042004-project Aug 27 02:31:33.569463 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-User-Domain-Id: default Aug 27 02:31:33.569463 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-User-Domain-Name: Default Aug 27 02:31:33.569463 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-User-Id: d945d512ae034ec08838d0a3afde2429 Aug 27 02:31:33.569463 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-User-Name: tempest-ServerRescueNegativeTestJSON-1551042004-project Aug 27 02:31:33.569463 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: method: > Aug 27 02:31:33.569463 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: exception: Instance 98e4cfeb-d49e-4888-a2c0-40a9e8af90d3 could not be found. Aug 27 02:31:33.569463 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: action_args: {'id': '98e4cfeb-d49e-4888-a2c0-40a9e8af90d3'} Aug 27 02:31:33.570291 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Aug 27 02:31:33.570291 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 27 02:31:33.570291 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Aug 27 02:31:33.570291 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Aug 27 02:31:33.570291 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Aug 27 02:31:33.570291 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Aug 27 02:31:33.570291 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Aug 27 02:31:33.570291 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Aug 27 02:31:33.570291 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Aug 27 02:31:33.570291 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Aug 27 02:31:33.570291 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 98e4cfeb-d49e-4888-a2c0-40a9e8af90d3 could not be found. Aug 27 02:31:33.570291 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi Aug 27 02:31:33.570291 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 27 02:31:33.570291 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi Aug 27 02:31:33.570291 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 27 02:31:33.570291 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 27 02:31:33.570291 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 27 02:31:33.570291 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Aug 27 02:31:33.570291 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Aug 27 02:31:33.570291 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Aug 27 02:31:33.571969 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Aug 27 02:31:33.571969 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Aug 27 02:31:33.571969 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi instance = self._get_server( Aug 27 02:31:33.571969 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Aug 27 02:31:33.571969 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Aug 27 02:31:33.571969 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Aug 27 02:31:33.571969 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Aug 27 02:31:33.571969 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 98e4cfeb-d49e-4888-a2c0-40a9e8af90d3 could not be found. Aug 27 02:31:33.571969 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi Aug 27 02:31:33.571969 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 27 02:31:33.571969 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi Aug 27 02:31:33.571969 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 27 02:31:33.571969 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 27 02:31:33.571969 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 27 02:31:33.571969 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Aug 27 02:31:33.571969 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Aug 27 02:31:33.571969 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 98e4cfeb-d49e-4888-a2c0-40a9e8af90d3 could not be found. Aug 27 02:31:33.571969 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi Aug 27 02:31:33.573519 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-1b1bac7a-3e13-4455-bb72-6daca9410ae1 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Returning 404 to user: Instance 98e4cfeb-d49e-4888-a2c0-40a9e8af90d3 could not be found. {{(pid=120863) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Aug 27 02:31:33.573519 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-1b1bac7a-3e13-4455-bb72-6daca9410ae1 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] 10.222.0.52 "GET /compute/v2.1/servers/98e4cfeb-d49e-4888-a2c0-40a9e8af90d3" status: 404 len: 111 microversion: 2.1 time: 0.033349 Aug 27 02:31:33.573519 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 424/853] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:31:33 2022] GET /compute/v2.1/servers/98e4cfeb-d49e-4888-a2c0-40a9e8af90d3 => generated 111 bytes in 35 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Aug 27 02:31:33.582068 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-670181f4-eb72-4ac7-85d1-52992d7aff0a tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Calling method '.version_select at 0x7f6b1b29e670>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:31:33.606553 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-670181f4-eb72-4ac7-85d1-52992d7aff0a tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] 10.222.0.52 "DELETE /compute/v2.1/os-keypairs/tempest-keypair-341970461" status: 202 len: 0 microversion: 2.1 time: 0.030243 Aug 27 02:31:33.607279 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 430/854] 10.222.0.52 () {58 vars in 1218 bytes} [Sat Aug 27 02:31:33 2022] DELETE /compute/v2.1/os-keypairs/tempest-keypair-341970461 => generated 0 bytes in 32 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Aug 27 02:31:36.561537 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-bc24c4eb-34b1-4211-a1ba-b3cc5ec19197 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:31:36.563769 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-bc24c4eb-34b1-4211-a1ba-b3cc5ec19197 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] [instance: 05bed8e9-f530-418a-9819-4f65dabdcda0] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:31:36.570360 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-bc24c4eb-34b1-4211-a1ba-b3cc5ec19197 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:31:36.570854 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-bc24c4eb-34b1-4211-a1ba-b3cc5ec19197 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:31:36.571292 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-bc24c4eb-34b1-4211-a1ba-b3cc5ec19197 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:31:36.571998 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-bc24c4eb-34b1-4211-a1ba-b3cc5ec19197 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:31:36.572454 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-bc24c4eb-34b1-4211-a1ba-b3cc5ec19197 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:31:36.572880 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-bc24c4eb-34b1-4211-a1ba-b3cc5ec19197 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:31:36.602934 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-bc24c4eb-34b1-4211-a1ba-b3cc5ec19197 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] [instance: 05bed8e9-f530-418a-9819-4f65dabdcda0] Going to try to terminate instance {{(pid=120863) delete /opt/stack/nova/nova/compute/api.py:2719}} Aug 27 02:31:36.704362 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-bc24c4eb-34b1-4211-a1ba-b3cc5ec19197 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] 10.222.0.52 "DELETE /compute/v2.1/servers/05bed8e9-f530-418a-9819-4f65dabdcda0" status: 204 len: 0 microversion: 2.1 time: 0.145919 Aug 27 02:31:36.705174 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 425/855] 10.222.0.52 () {58 vars in 1239 bytes} [Sat Aug 27 02:31:36 2022] DELETE /compute/v2.1/servers/05bed8e9-f530-418a-9819-4f65dabdcda0 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Aug 27 02:31:36.713703 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-bb666db8-a572-42f4-9d0e-24a9d05248f7 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Calling method '>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:31:36.714662 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-bb666db8-a572-42f4-9d0e-24a9d05248f7 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] [instance: 05bed8e9-f530-418a-9819-4f65dabdcda0] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:31:36.722328 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-bb666db8-a572-42f4-9d0e-24a9d05248f7 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:31:36.722969 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-bb666db8-a572-42f4-9d0e-24a9d05248f7 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:31:36.723411 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-bb666db8-a572-42f4-9d0e-24a9d05248f7 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:31:36.724115 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-bb666db8-a572-42f4-9d0e-24a9d05248f7 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:31:36.724601 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-bb666db8-a572-42f4-9d0e-24a9d05248f7 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:31:36.725028 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-bb666db8-a572-42f4-9d0e-24a9d05248f7 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:31:36.875947 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-bb666db8-a572-42f4-9d0e-24a9d05248f7 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd945d512ae034ec08838d0a3afde2429', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '880ac4003248441c809a76cfd1b221eb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:31:36.887568 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-bb666db8-a572-42f4-9d0e-24a9d05248f7 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] 10.222.0.52 "GET /compute/v2.1/servers/05bed8e9-f530-418a-9819-4f65dabdcda0" status: 200 len: 1507 microversion: 2.1 time: 0.178091 Aug 27 02:31:36.888186 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 431/856] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:31:36 2022] GET /compute/v2.1/servers/05bed8e9-f530-418a-9819-4f65dabdcda0 => generated 1507 bytes in 179 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:31:37.910534 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-9b22c8cb-b40e-431b-83a5-ee51217d974c tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:31:37.911152 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-9b22c8cb-b40e-431b-83a5-ee51217d974c tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] [instance: 05bed8e9-f530-418a-9819-4f65dabdcda0] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:31:37.918089 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-9b22c8cb-b40e-431b-83a5-ee51217d974c tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:31:37.918581 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-9b22c8cb-b40e-431b-83a5-ee51217d974c tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:31:37.919019 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-9b22c8cb-b40e-431b-83a5-ee51217d974c tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:31:37.919723 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-9b22c8cb-b40e-431b-83a5-ee51217d974c tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:31:37.920176 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-9b22c8cb-b40e-431b-83a5-ee51217d974c tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:31:37.920601 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-9b22c8cb-b40e-431b-83a5-ee51217d974c tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:31:38.061598 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-9b22c8cb-b40e-431b-83a5-ee51217d974c tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd945d512ae034ec08838d0a3afde2429', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '880ac4003248441c809a76cfd1b221eb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:31:38.070243 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-9b22c8cb-b40e-431b-83a5-ee51217d974c tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] 10.222.0.52 "GET /compute/v2.1/servers/05bed8e9-f530-418a-9819-4f65dabdcda0" status: 200 len: 1507 microversion: 2.1 time: 0.163866 Aug 27 02:31:38.070906 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 426/857] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:31:37 2022] GET /compute/v2.1/servers/05bed8e9-f530-418a-9819-4f65dabdcda0 => generated 1507 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:31:38.363252 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [req-cebbbda9-4a2c-48cd-8bcc-af26151d79b9 req-8eb662e4-77d0-468d-b713-9dcd898d57bb service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "05bed8e9-f530-418a-9819-4f65dabdcda0", "name": "network-vif-deleted", "tag": "81921b15-5b92-41f8-a6fa-30be932b9435"}]} {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 27 02:31:38.373515 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [req-cebbbda9-4a2c-48cd-8bcc-af26151d79b9 req-8eb662e4-77d0-468d-b713-9dcd898d57bb service nova] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:31:38.373986 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [req-cebbbda9-4a2c-48cd-8bcc-af26151d79b9 req-8eb662e4-77d0-468d-b713-9dcd898d57bb service nova] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:31:38.374413 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [req-cebbbda9-4a2c-48cd-8bcc-af26151d79b9 req-8eb662e4-77d0-468d-b713-9dcd898d57bb service nova] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:31:38.386830 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.compute.server_external_events [req-cebbbda9-4a2c-48cd-8bcc-af26151d79b9 req-8eb662e4-77d0-468d-b713-9dcd898d57bb service nova] Creating event network-vif-deleted:81921b15-5b92-41f8-a6fa-30be932b9435 for instance 05bed8e9-f530-418a-9819-4f65dabdcda0 on hv-cinder-84630 Aug 27 02:31:38.391689 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [req-cebbbda9-4a2c-48cd-8bcc-af26151d79b9 req-8eb662e4-77d0-468d-b713-9dcd898d57bb service nova] 10.222.0.52 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.032039 Aug 27 02:31:38.392288 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 432/858] 10.222.0.52 () {64 vars in 1317 bytes} [Sat Aug 27 02:31:38 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 33 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 27 02:31:39.093700 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-d065e67a-8fc0-4d67-8367-a9c95786ab0d tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:31:39.094367 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-d065e67a-8fc0-4d67-8367-a9c95786ab0d tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] [instance: 05bed8e9-f530-418a-9819-4f65dabdcda0] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:31:39.101099 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-d065e67a-8fc0-4d67-8367-a9c95786ab0d tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:31:39.101613 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-d065e67a-8fc0-4d67-8367-a9c95786ab0d tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:31:39.102063 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-d065e67a-8fc0-4d67-8367-a9c95786ab0d tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:31:39.102782 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-d065e67a-8fc0-4d67-8367-a9c95786ab0d tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:31:39.103249 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-d065e67a-8fc0-4d67-8367-a9c95786ab0d tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:31:39.103673 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-d065e67a-8fc0-4d67-8367-a9c95786ab0d tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:31:39.136224 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-d065e67a-8fc0-4d67-8367-a9c95786ab0d tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:31:39.136730 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-d065e67a-8fc0-4d67-8367-a9c95786ab0d tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:31:39.137184 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-d065e67a-8fc0-4d67-8367-a9c95786ab0d tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:31:39.137631 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.objects.instance [None req-d065e67a-8fc0-4d67-8367-a9c95786ab0d tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lazy-loading 'fault' on Instance uuid 05bed8e9-f530-418a-9819-4f65dabdcda0 {{(pid=120863) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Aug 27 02:31:39.188718 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-d065e67a-8fc0-4d67-8367-a9c95786ab0d tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd945d512ae034ec08838d0a3afde2429', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '880ac4003248441c809a76cfd1b221eb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:31:39.201022 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-d065e67a-8fc0-4d67-8367-a9c95786ab0d tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] 10.222.0.52 "GET /compute/v2.1/servers/05bed8e9-f530-418a-9819-4f65dabdcda0" status: 200 len: 1312 microversion: 2.1 time: 0.112051 Aug 27 02:31:39.201648 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 427/859] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:31:39 2022] GET /compute/v2.1/servers/05bed8e9-f530-418a-9819-4f65dabdcda0 => generated 1312 bytes in 113 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:31:40.227293 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-cd975140-c8a5-4860-9604-b25c73fead3c tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Calling method '>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:31:40.228436 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-cd975140-c8a5-4860-9604-b25c73fead3c tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] [instance: 05bed8e9-f530-418a-9819-4f65dabdcda0] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:31:40.235795 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-cd975140-c8a5-4860-9604-b25c73fead3c tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:31:40.236273 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-cd975140-c8a5-4860-9604-b25c73fead3c tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:31:40.236700 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-cd975140-c8a5-4860-9604-b25c73fead3c tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:31:40.237591 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-cd975140-c8a5-4860-9604-b25c73fead3c tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:31:40.238039 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-cd975140-c8a5-4860-9604-b25c73fead3c tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:31:40.238462 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-cd975140-c8a5-4860-9604-b25c73fead3c tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:31:40.252772 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.wsgi [None req-cd975140-c8a5-4860-9604-b25c73fead3c tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] HTTP exception thrown: Instance 05bed8e9-f530-418a-9819-4f65dabdcda0 could not be found. Aug 27 02:31:40.254451 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-cd975140-c8a5-4860-9604-b25c73fead3c tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Request method failure captured: Aug 27 02:31:40.254451 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: request: GET /compute/v2.1/servers/05bed8e9-f530-418a-9819-4f65dabdcda0 HTTP/1.1 Aug 27 02:31:40.254451 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: Accept: application/json Aug 27 02:31:40.254451 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: Accept-Encoding: identity Aug 27 02:31:40.254451 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: Connection: close Aug 27 02:31:40.254451 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: Content-Length: 0 Aug 27 02:31:40.254451 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: Content-Type: application/json Aug 27 02:31:40.254451 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: Host: 10.222.0.52 Aug 27 02:31:40.254451 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: Openstack-System-Scope: None Aug 27 02:31:40.254451 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: User-Agent: python-urllib3/1.26.11 Aug 27 02:31:40.254451 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-Auth-Token: gAAAAABjCX_mTgf5582eaPxg7VeVEFNV70u35gVl4tYbLlqoLLSOTdT51sTqNUwxXgJqsRZlTyjbNrsB5eN_l1XXJng0l2G5rQDKOgGEffO457OUUMgEUK1evfhiyjkwW8ZvB7sHsdxG4zKRGU2G452QU2-KddZbqwFS3zX2uIy2nVW08GLJmCc Aug 27 02:31:40.254451 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-Domain-Id: None Aug 27 02:31:40.254451 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-Domain-Name: None Aug 27 02:31:40.254451 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-Identity-Status: Confirmed Aug 27 02:31:40.254451 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-Is-Admin-Project: True Aug 27 02:31:40.254451 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-Project-Domain-Id: default Aug 27 02:31:40.254451 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-Project-Domain-Name: Default Aug 27 02:31:40.254451 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-Project-Id: 880ac4003248441c809a76cfd1b221eb Aug 27 02:31:40.254451 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-Project-Name: tempest-ServerRescueNegativeTestJSON-1551042004 Aug 27 02:31:40.254451 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-Role: reader,member Aug 27 02:31:40.254451 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-Roles: reader,member Aug 27 02:31:40.256022 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-Service-Catalog: [{"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.52:8080", "publicURL": "http://10.222.0.52:8080/v1/AUTH_880ac4003248441c809a76cfd1b221eb"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52:9696/networking"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/compute/v2/880ac4003248441c809a76cfd1b221eb"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/image"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/volume/v3/880ac4003248441c809a76cfd1b221eb"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/identity"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/volume/v3/880ac4003248441c809a76cfd1b221eb"}]}] Aug 27 02:31:40.256022 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-Tenant: tempest-ServerRescueNegativeTestJSON-1551042004 Aug 27 02:31:40.256022 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-Tenant-Id: 880ac4003248441c809a76cfd1b221eb Aug 27 02:31:40.256022 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-Tenant-Name: tempest-ServerRescueNegativeTestJSON-1551042004 Aug 27 02:31:40.256022 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-User: tempest-ServerRescueNegativeTestJSON-1551042004-project Aug 27 02:31:40.256022 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-User-Domain-Id: default Aug 27 02:31:40.256022 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-User-Domain-Name: Default Aug 27 02:31:40.256022 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-User-Id: d945d512ae034ec08838d0a3afde2429 Aug 27 02:31:40.256022 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-User-Name: tempest-ServerRescueNegativeTestJSON-1551042004-project Aug 27 02:31:40.256022 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: method: > Aug 27 02:31:40.256022 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: exception: Instance 05bed8e9-f530-418a-9819-4f65dabdcda0 could not be found. Aug 27 02:31:40.256022 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: action_args: {'id': '05bed8e9-f530-418a-9819-4f65dabdcda0'} Aug 27 02:31:40.256983 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Aug 27 02:31:40.256983 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 27 02:31:40.256983 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Aug 27 02:31:40.256983 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Aug 27 02:31:40.256983 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Aug 27 02:31:40.256983 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Aug 27 02:31:40.256983 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Aug 27 02:31:40.256983 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Aug 27 02:31:40.256983 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Aug 27 02:31:40.256983 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Aug 27 02:31:40.256983 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 05bed8e9-f530-418a-9819-4f65dabdcda0 could not be found. Aug 27 02:31:40.256983 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi Aug 27 02:31:40.256983 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 27 02:31:40.256983 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi Aug 27 02:31:40.256983 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 27 02:31:40.256983 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 27 02:31:40.256983 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 27 02:31:40.256983 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Aug 27 02:31:40.256983 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Aug 27 02:31:40.256983 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Aug 27 02:31:40.258600 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Aug 27 02:31:40.258600 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Aug 27 02:31:40.258600 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi instance = self._get_server( Aug 27 02:31:40.258600 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Aug 27 02:31:40.258600 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Aug 27 02:31:40.258600 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Aug 27 02:31:40.258600 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Aug 27 02:31:40.258600 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 05bed8e9-f530-418a-9819-4f65dabdcda0 could not be found. Aug 27 02:31:40.258600 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi Aug 27 02:31:40.258600 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 27 02:31:40.258600 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi Aug 27 02:31:40.258600 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 27 02:31:40.258600 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 27 02:31:40.258600 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 27 02:31:40.258600 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Aug 27 02:31:40.258600 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Aug 27 02:31:40.258600 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 05bed8e9-f530-418a-9819-4f65dabdcda0 could not be found. Aug 27 02:31:40.258600 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi Aug 27 02:31:40.260025 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-cd975140-c8a5-4860-9604-b25c73fead3c tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Returning 404 to user: Instance 05bed8e9-f530-418a-9819-4f65dabdcda0 could not be found. {{(pid=120864) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Aug 27 02:31:40.260025 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-cd975140-c8a5-4860-9604-b25c73fead3c tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] 10.222.0.52 "GET /compute/v2.1/servers/05bed8e9-f530-418a-9819-4f65dabdcda0" status: 404 len: 111 microversion: 2.1 time: 0.035117 Aug 27 02:31:40.260025 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 433/860] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:31:40 2022] GET /compute/v2.1/servers/05bed8e9-f530-418a-9819-4f65dabdcda0 => generated 111 bytes in 36 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Aug 27 02:31:40.265075 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-483326a3-77d8-4db8-9fd4-cda8c5e9d23b tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:31:40.265628 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-483326a3-77d8-4db8-9fd4-cda8c5e9d23b tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] [instance: e2bf7024-0779-4e08-844a-43b78187e770] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:31:40.272572 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-483326a3-77d8-4db8-9fd4-cda8c5e9d23b tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:31:40.273075 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-483326a3-77d8-4db8-9fd4-cda8c5e9d23b tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:31:40.273574 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-483326a3-77d8-4db8-9fd4-cda8c5e9d23b tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:31:40.274260 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-483326a3-77d8-4db8-9fd4-cda8c5e9d23b tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:31:40.274712 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-483326a3-77d8-4db8-9fd4-cda8c5e9d23b tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:31:40.275161 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-483326a3-77d8-4db8-9fd4-cda8c5e9d23b tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:31:40.302640 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-483326a3-77d8-4db8-9fd4-cda8c5e9d23b tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] [instance: e2bf7024-0779-4e08-844a-43b78187e770] Going to try to terminate instance {{(pid=120863) delete /opt/stack/nova/nova/compute/api.py:2719}} Aug 27 02:31:40.396712 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-483326a3-77d8-4db8-9fd4-cda8c5e9d23b tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] 10.222.0.52 "DELETE /compute/v2.1/servers/e2bf7024-0779-4e08-844a-43b78187e770" status: 204 len: 0 microversion: 2.1 time: 0.135444 Aug 27 02:31:40.397571 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 428/861] 10.222.0.52 () {58 vars in 1239 bytes} [Sat Aug 27 02:31:40 2022] DELETE /compute/v2.1/servers/e2bf7024-0779-4e08-844a-43b78187e770 => generated 0 bytes in 137 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Aug 27 02:31:40.407259 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-ac44bcc1-6275-4611-8174-a357b2ef9fb7 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Calling method '>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:31:40.407801 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-ac44bcc1-6275-4611-8174-a357b2ef9fb7 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] [instance: e2bf7024-0779-4e08-844a-43b78187e770] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:31:40.414265 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-ac44bcc1-6275-4611-8174-a357b2ef9fb7 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:31:40.414751 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-ac44bcc1-6275-4611-8174-a357b2ef9fb7 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:31:40.415212 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-ac44bcc1-6275-4611-8174-a357b2ef9fb7 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:31:40.415880 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-ac44bcc1-6275-4611-8174-a357b2ef9fb7 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:31:40.416373 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-ac44bcc1-6275-4611-8174-a357b2ef9fb7 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:31:40.416766 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-ac44bcc1-6275-4611-8174-a357b2ef9fb7 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:31:40.554501 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-ac44bcc1-6275-4611-8174-a357b2ef9fb7 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd945d512ae034ec08838d0a3afde2429', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '880ac4003248441c809a76cfd1b221eb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:31:40.563250 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-ac44bcc1-6275-4611-8174-a357b2ef9fb7 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] 10.222.0.52 "GET /compute/v2.1/servers/e2bf7024-0779-4e08-844a-43b78187e770" status: 200 len: 1522 microversion: 2.1 time: 0.159838 Aug 27 02:31:40.563829 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 434/862] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:31:40 2022] GET /compute/v2.1/servers/e2bf7024-0779-4e08-844a-43b78187e770 => generated 1522 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:31:41.586231 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-5b730052-a5a8-46bd-bc88-a383aed863a5 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:31:41.586962 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-5b730052-a5a8-46bd-bc88-a383aed863a5 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] [instance: e2bf7024-0779-4e08-844a-43b78187e770] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:31:41.596644 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-5b730052-a5a8-46bd-bc88-a383aed863a5 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:31:41.597448 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-5b730052-a5a8-46bd-bc88-a383aed863a5 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:31:41.597965 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-5b730052-a5a8-46bd-bc88-a383aed863a5 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:31:41.599572 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-5b730052-a5a8-46bd-bc88-a383aed863a5 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:31:41.600836 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-5b730052-a5a8-46bd-bc88-a383aed863a5 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:31:41.601394 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-5b730052-a5a8-46bd-bc88-a383aed863a5 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:31:41.767367 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-5b730052-a5a8-46bd-bc88-a383aed863a5 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd945d512ae034ec08838d0a3afde2429', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '880ac4003248441c809a76cfd1b221eb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:31:41.776143 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-5b730052-a5a8-46bd-bc88-a383aed863a5 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] 10.222.0.52 "GET /compute/v2.1/servers/e2bf7024-0779-4e08-844a-43b78187e770" status: 200 len: 1522 microversion: 2.1 time: 0.194285 Aug 27 02:31:41.776473 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [req-cfd3efe7-f13f-4f6a-874c-a4abc36fddc1 req-e1763558-28ea-4f81-919a-0b5b73f1fe12 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "e2bf7024-0779-4e08-844a-43b78187e770", "name": "network-vif-deleted", "tag": "66f55f99-dc75-425b-9214-a2edc384e389"}]} {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 27 02:31:41.777308 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 429/863] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:31:41 2022] GET /compute/v2.1/servers/e2bf7024-0779-4e08-844a-43b78187e770 => generated 1522 bytes in 196 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:31:41.789802 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [req-cfd3efe7-f13f-4f6a-874c-a4abc36fddc1 req-e1763558-28ea-4f81-919a-0b5b73f1fe12 service nova] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:31:41.790279 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [req-cfd3efe7-f13f-4f6a-874c-a4abc36fddc1 req-e1763558-28ea-4f81-919a-0b5b73f1fe12 service nova] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:31:41.790712 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [req-cfd3efe7-f13f-4f6a-874c-a4abc36fddc1 req-e1763558-28ea-4f81-919a-0b5b73f1fe12 service nova] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:31:41.805058 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.compute.server_external_events [req-cfd3efe7-f13f-4f6a-874c-a4abc36fddc1 req-e1763558-28ea-4f81-919a-0b5b73f1fe12 service nova] Creating event network-vif-deleted:66f55f99-dc75-425b-9214-a2edc384e389 for instance e2bf7024-0779-4e08-844a-43b78187e770 on hv-cinder-84630 Aug 27 02:31:41.813666 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [req-cfd3efe7-f13f-4f6a-874c-a4abc36fddc1 req-e1763558-28ea-4f81-919a-0b5b73f1fe12 service nova] 10.222.0.52 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.041993 Aug 27 02:31:41.814279 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 435/864] 10.222.0.52 () {64 vars in 1317 bytes} [Sat Aug 27 02:31:41 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 27 02:31:42.799120 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-21157825-01ac-407e-94c9-2ba099843833 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:31:42.799733 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-21157825-01ac-407e-94c9-2ba099843833 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] [instance: e2bf7024-0779-4e08-844a-43b78187e770] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:31:42.808404 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-21157825-01ac-407e-94c9-2ba099843833 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:31:42.808750 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-21157825-01ac-407e-94c9-2ba099843833 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:31:42.809224 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-21157825-01ac-407e-94c9-2ba099843833 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:31:42.809921 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-21157825-01ac-407e-94c9-2ba099843833 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:31:42.810377 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-21157825-01ac-407e-94c9-2ba099843833 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:31:42.810809 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-21157825-01ac-407e-94c9-2ba099843833 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:31:42.825928 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.wsgi [None req-21157825-01ac-407e-94c9-2ba099843833 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] HTTP exception thrown: Instance e2bf7024-0779-4e08-844a-43b78187e770 could not be found. Aug 27 02:31:42.827591 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-21157825-01ac-407e-94c9-2ba099843833 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Request method failure captured: Aug 27 02:31:42.827591 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: request: GET /compute/v2.1/servers/e2bf7024-0779-4e08-844a-43b78187e770 HTTP/1.1 Aug 27 02:31:42.827591 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: Accept: application/json Aug 27 02:31:42.827591 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: Accept-Encoding: identity Aug 27 02:31:42.827591 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: Connection: close Aug 27 02:31:42.827591 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: Content-Length: 0 Aug 27 02:31:42.827591 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: Content-Type: application/json Aug 27 02:31:42.827591 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: Host: 10.222.0.52 Aug 27 02:31:42.827591 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: Openstack-System-Scope: None Aug 27 02:31:42.827591 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: User-Agent: python-urllib3/1.26.11 Aug 27 02:31:42.827591 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-Auth-Token: gAAAAABjCX_mTgf5582eaPxg7VeVEFNV70u35gVl4tYbLlqoLLSOTdT51sTqNUwxXgJqsRZlTyjbNrsB5eN_l1XXJng0l2G5rQDKOgGEffO457OUUMgEUK1evfhiyjkwW8ZvB7sHsdxG4zKRGU2G452QU2-KddZbqwFS3zX2uIy2nVW08GLJmCc Aug 27 02:31:42.827591 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-Domain-Id: None Aug 27 02:31:42.827591 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-Domain-Name: None Aug 27 02:31:42.827591 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-Identity-Status: Confirmed Aug 27 02:31:42.827591 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-Is-Admin-Project: True Aug 27 02:31:42.827591 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-Project-Domain-Id: default Aug 27 02:31:42.827591 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-Project-Domain-Name: Default Aug 27 02:31:42.827591 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-Project-Id: 880ac4003248441c809a76cfd1b221eb Aug 27 02:31:42.827591 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-Project-Name: tempest-ServerRescueNegativeTestJSON-1551042004 Aug 27 02:31:42.827591 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-Role: reader,member Aug 27 02:31:42.827591 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-Roles: reader,member Aug 27 02:31:42.829176 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-Service-Catalog: [{"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.52:8080", "publicURL": "http://10.222.0.52:8080/v1/AUTH_880ac4003248441c809a76cfd1b221eb"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52:9696/networking"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/compute/v2/880ac4003248441c809a76cfd1b221eb"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/image"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/volume/v3/880ac4003248441c809a76cfd1b221eb"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/identity"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/volume/v3/880ac4003248441c809a76cfd1b221eb"}]}] Aug 27 02:31:42.829176 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-Tenant: tempest-ServerRescueNegativeTestJSON-1551042004 Aug 27 02:31:42.829176 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-Tenant-Id: 880ac4003248441c809a76cfd1b221eb Aug 27 02:31:42.829176 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-Tenant-Name: tempest-ServerRescueNegativeTestJSON-1551042004 Aug 27 02:31:42.829176 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-User: tempest-ServerRescueNegativeTestJSON-1551042004-project Aug 27 02:31:42.829176 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-User-Domain-Id: default Aug 27 02:31:42.829176 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-User-Domain-Name: Default Aug 27 02:31:42.829176 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-User-Id: d945d512ae034ec08838d0a3afde2429 Aug 27 02:31:42.829176 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-User-Name: tempest-ServerRescueNegativeTestJSON-1551042004-project Aug 27 02:31:42.829176 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: method: > Aug 27 02:31:42.829176 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: exception: Instance e2bf7024-0779-4e08-844a-43b78187e770 could not be found. Aug 27 02:31:42.829176 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: action_args: {'id': 'e2bf7024-0779-4e08-844a-43b78187e770'} Aug 27 02:31:42.829998 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Aug 27 02:31:42.829998 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 27 02:31:42.829998 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Aug 27 02:31:42.829998 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Aug 27 02:31:42.829998 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Aug 27 02:31:42.829998 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Aug 27 02:31:42.829998 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Aug 27 02:31:42.829998 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Aug 27 02:31:42.829998 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Aug 27 02:31:42.829998 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Aug 27 02:31:42.829998 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance e2bf7024-0779-4e08-844a-43b78187e770 could not be found. Aug 27 02:31:42.829998 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi Aug 27 02:31:42.829998 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 27 02:31:42.829998 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi Aug 27 02:31:42.829998 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 27 02:31:42.829998 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 27 02:31:42.829998 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 27 02:31:42.829998 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Aug 27 02:31:42.829998 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Aug 27 02:31:42.829998 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Aug 27 02:31:42.831703 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Aug 27 02:31:42.831703 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Aug 27 02:31:42.831703 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi instance = self._get_server( Aug 27 02:31:42.831703 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Aug 27 02:31:42.831703 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Aug 27 02:31:42.831703 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Aug 27 02:31:42.831703 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Aug 27 02:31:42.831703 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance e2bf7024-0779-4e08-844a-43b78187e770 could not be found. Aug 27 02:31:42.831703 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi Aug 27 02:31:42.831703 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 27 02:31:42.831703 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi Aug 27 02:31:42.831703 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 27 02:31:42.831703 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 27 02:31:42.831703 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 27 02:31:42.831703 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Aug 27 02:31:42.831703 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Aug 27 02:31:42.831703 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance e2bf7024-0779-4e08-844a-43b78187e770 could not be found. Aug 27 02:31:42.831703 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi Aug 27 02:31:42.832985 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-21157825-01ac-407e-94c9-2ba099843833 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] Returning 404 to user: Instance e2bf7024-0779-4e08-844a-43b78187e770 could not be found. {{(pid=120863) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Aug 27 02:31:42.832985 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-21157825-01ac-407e-94c9-2ba099843833 tempest-ServerRescueNegativeTestJSON-1551042004 tempest-ServerRescueNegativeTestJSON-1551042004-project] 10.222.0.52 "GET /compute/v2.1/servers/e2bf7024-0779-4e08-844a-43b78187e770" status: 404 len: 111 microversion: 2.1 time: 0.034533 Aug 27 02:31:42.832985 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 430/865] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:31:42 2022] GET /compute/v2.1/servers/e2bf7024-0779-4e08-844a-43b78187e770 => generated 111 bytes in 36 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Aug 27 02:31:47.704202 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-2baba2d6-add6-4f49-8256-7d4860ca98b2 tempest-VolumesGetTestJSON-514999675 tempest-VolumesGetTestJSON-514999675-project] Action: 'create', calling method: .version_select at 0x7f6b1e8cadc0>, body: {"volume": {"size": 1, "display_name": "tempest-VolumesGetTestJSON-Volume-1012242095", "metadata": {"Type": "work"}}} {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 27 02:31:48.027866 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-2baba2d6-add6-4f49-8256-7d4860ca98b2 tempest-VolumesGetTestJSON-514999675 tempest-VolumesGetTestJSON-514999675-project] 10.222.0.52 "POST /compute/v2.1/os-volumes" status: 200 len: 353 microversion: 2.1 time: 0.485457 Aug 27 02:31:48.028662 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 436/866] 10.222.0.52 () {60 vars in 1156 bytes} [Sat Aug 27 02:31:47 2022] POST /compute/v2.1/os-volumes => generated 353 bytes in 487 msecs (HTTP/1.1 200) 10 headers in 448 bytes (2 switches on core 0) Aug 27 02:31:48.042121 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-ab094e4a-ac02-4c52-8aa2-adfe753d290c tempest-VolumesGetTestJSON-514999675 tempest-VolumesGetTestJSON-514999675-project] Calling method '.version_select at 0x7f6b1b01ec10>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:31:48.099125 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-ab094e4a-ac02-4c52-8aa2-adfe753d290c tempest-VolumesGetTestJSON-514999675 tempest-VolumesGetTestJSON-514999675-project] 10.222.0.52 "GET /compute/v2.1/os-volumes/0440b424-9658-446b-994e-4e0b2de524d7" status: 200 len: 353 microversion: 2.1 time: 0.061149 Aug 27 02:31:48.102476 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 431/867] 10.222.0.52 () {58 vars in 1245 bytes} [Sat Aug 27 02:31:48 2022] GET /compute/v2.1/os-volumes/0440b424-9658-446b-994e-4e0b2de524d7 => generated 353 bytes in 66 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 27 02:31:49.119241 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-a0f53fa6-baa0-41cd-961a-816caeb44138 tempest-VolumesGetTestJSON-514999675 tempest-VolumesGetTestJSON-514999675-project] Calling method '.version_select at 0x7f6b1ea10670>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:31:49.173842 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-a0f53fa6-baa0-41cd-961a-816caeb44138 tempest-VolumesGetTestJSON-514999675 tempest-VolumesGetTestJSON-514999675-project] 10.222.0.52 "GET /compute/v2.1/os-volumes/0440b424-9658-446b-994e-4e0b2de524d7" status: 200 len: 354 microversion: 2.1 time: 0.059873 Aug 27 02:31:49.174408 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 437/868] 10.222.0.52 () {58 vars in 1245 bytes} [Sat Aug 27 02:31:49 2022] GET /compute/v2.1/os-volumes/0440b424-9658-446b-994e-4e0b2de524d7 => generated 354 bytes in 61 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 27 02:31:49.187949 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-8be16d93-f60f-4688-9ada-83cff62b8e21 tempest-VolumesGetTestJSON-514999675 tempest-VolumesGetTestJSON-514999675-project] Calling method '.version_select at 0x7f6b1b0e2700>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:31:49.236505 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-8be16d93-f60f-4688-9ada-83cff62b8e21 tempest-VolumesGetTestJSON-514999675 tempest-VolumesGetTestJSON-514999675-project] 10.222.0.52 "GET /compute/v2.1/os-volumes/0440b424-9658-446b-994e-4e0b2de524d7" status: 200 len: 354 microversion: 2.1 time: 0.052845 Aug 27 02:31:49.237062 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 432/869] 10.222.0.52 () {58 vars in 1245 bytes} [Sat Aug 27 02:31:49 2022] GET /compute/v2.1/os-volumes/0440b424-9658-446b-994e-4e0b2de524d7 => generated 354 bytes in 54 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 27 02:31:49.250923 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-30fba227-369b-4093-bb91-7c1378a1f553 tempest-VolumesGetTestJSON-514999675 tempest-VolumesGetTestJSON-514999675-project] Calling method '.version_select at 0x7f6b1b49e700>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:31:49.317228 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-30fba227-369b-4093-bb91-7c1378a1f553 tempest-VolumesGetTestJSON-514999675 tempest-VolumesGetTestJSON-514999675-project] 10.222.0.52 "DELETE /compute/v2.1/os-volumes/0440b424-9658-446b-994e-4e0b2de524d7" status: 202 len: 0 microversion: 2.1 time: 0.069999 Aug 27 02:31:49.317806 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 438/870] 10.222.0.52 () {58 vars in 1248 bytes} [Sat Aug 27 02:31:49 2022] DELETE /compute/v2.1/os-volumes/0440b424-9658-446b-994e-4e0b2de524d7 => generated 0 bytes in 71 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Aug 27 02:31:49.326215 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-25d9f943-68d4-422c-9936-7ca102d48389 tempest-VolumesGetTestJSON-514999675 tempest-VolumesGetTestJSON-514999675-project] Calling method '.version_select at 0x7f6b1b035280>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:31:49.377534 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-25d9f943-68d4-422c-9936-7ca102d48389 tempest-VolumesGetTestJSON-514999675 tempest-VolumesGetTestJSON-514999675-project] 10.222.0.52 "GET /compute/v2.1/os-volumes/0440b424-9658-446b-994e-4e0b2de524d7" status: 200 len: 353 microversion: 2.1 time: 0.055211 Aug 27 02:31:49.378105 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 433/871] 10.222.0.52 () {58 vars in 1245 bytes} [Sat Aug 27 02:31:49 2022] GET /compute/v2.1/os-volumes/0440b424-9658-446b-994e-4e0b2de524d7 => generated 353 bytes in 56 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 27 02:31:50.392080 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-f82f55f0-f79e-409a-bdfc-78c0efa2ba72 tempest-VolumesGetTestJSON-514999675 tempest-VolumesGetTestJSON-514999675-project] Calling method '.version_select at 0x7f6b1e8ca310>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:31:50.420983 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.wsgi [None req-f82f55f0-f79e-409a-bdfc-78c0efa2ba72 tempest-VolumesGetTestJSON-514999675 tempest-VolumesGetTestJSON-514999675-project] HTTP exception thrown: Volume 0440b424-9658-446b-994e-4e0b2de524d7 could not be found. Aug 27 02:31:50.422786 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-f82f55f0-f79e-409a-bdfc-78c0efa2ba72 tempest-VolumesGetTestJSON-514999675 tempest-VolumesGetTestJSON-514999675-project] Request method failure captured: Aug 27 02:31:50.422786 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: request: GET /compute/v2.1/os-volumes/0440b424-9658-446b-994e-4e0b2de524d7 HTTP/1.1 Aug 27 02:31:50.422786 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: Accept: application/json Aug 27 02:31:50.422786 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: Accept-Encoding: identity Aug 27 02:31:50.422786 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: Connection: close Aug 27 02:31:50.422786 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: Content-Length: 0 Aug 27 02:31:50.422786 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: Content-Type: application/json Aug 27 02:31:50.422786 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: Host: 10.222.0.52 Aug 27 02:31:50.422786 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: Openstack-System-Scope: None Aug 27 02:31:50.422786 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: User-Agent: python-urllib3/1.26.11 Aug 27 02:31:50.422786 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-Auth-Token: gAAAAABjCYITdSyU6rvOx11JamrgXREfHYgkOE7KzDnfTzaiM5WB-mt8RtRjWf8LIhlLQZ7r_wrbHxe0vFNqTg5J529EO5-tcmwzMGC_H4qa_oZYdrABR6kBgnMrJVK6IqO-7odYnY8RrFyKpBoZLtCkh192A4ufez_nmM2tUKQib5L4PjnljKs Aug 27 02:31:50.422786 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-Domain-Id: None Aug 27 02:31:50.422786 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-Domain-Name: None Aug 27 02:31:50.422786 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-Identity-Status: Confirmed Aug 27 02:31:50.422786 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-Is-Admin-Project: True Aug 27 02:31:50.422786 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-Project-Domain-Id: default Aug 27 02:31:50.422786 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-Project-Domain-Name: Default Aug 27 02:31:50.422786 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-Project-Id: 21f532fa34834346bcceca9a40a7fc09 Aug 27 02:31:50.422786 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-Project-Name: tempest-VolumesGetTestJSON-514999675 Aug 27 02:31:50.422786 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-Role: reader,member Aug 27 02:31:50.422786 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-Roles: reader,member Aug 27 02:31:50.426774 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-Service-Catalog: [{"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.52:8080", "publicURL": "http://10.222.0.52:8080/v1/AUTH_21f532fa34834346bcceca9a40a7fc09"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52:9696/networking"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/compute/v2/21f532fa34834346bcceca9a40a7fc09"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/image"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/volume/v3/21f532fa34834346bcceca9a40a7fc09"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/identity"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/volume/v3/21f532fa34834346bcceca9a40a7fc09"}]}] Aug 27 02:31:50.426774 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-Tenant: tempest-VolumesGetTestJSON-514999675 Aug 27 02:31:50.426774 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-Tenant-Id: 21f532fa34834346bcceca9a40a7fc09 Aug 27 02:31:50.426774 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-Tenant-Name: tempest-VolumesGetTestJSON-514999675 Aug 27 02:31:50.426774 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-User: tempest-VolumesGetTestJSON-514999675-project Aug 27 02:31:50.426774 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-User-Domain-Id: default Aug 27 02:31:50.426774 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-User-Domain-Name: Default Aug 27 02:31:50.426774 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-User-Id: ccdb258153c84117aa02b27aa8e79a79 Aug 27 02:31:50.426774 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-User-Name: tempest-VolumesGetTestJSON-514999675-project Aug 27 02:31:50.426774 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: method: Aug 27 02:31:50.426774 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: exception: Volume 0440b424-9658-446b-994e-4e0b2de524d7 could not be found. Aug 27 02:31:50.426774 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: action_args: {'id': '0440b424-9658-446b-994e-4e0b2de524d7'} Aug 27 02:31:50.426774 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Aug 27 02:31:50.429490 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 27 02:31:50.429490 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Aug 27 02:31:50.429490 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Aug 27 02:31:50.429490 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 502, in get Aug 27 02:31:50.429490 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi item = cinderclient( Aug 27 02:31:50.429490 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 202, in get Aug 27 02:31:50.429490 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi return self._get("/volumes/%s" % volume_id, "volume") Aug 27 02:31:50.429490 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 297, in _get Aug 27 02:31:50.429490 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.get(url) Aug 27 02:31:50.429490 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 224, in get Aug 27 02:31:50.429490 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'GET', **kwargs) Aug 27 02:31:50.429490 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Aug 27 02:31:50.429490 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Aug 27 02:31:50.429490 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Aug 27 02:31:50.429490 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Aug 27 02:31:50.429490 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.NotFound: Volume 0440b424-9658-446b-994e-4e0b2de524d7 could not be found. (HTTP 404) (Request-ID: req-7d7e0816-1702-4deb-9860-307d111f0bcb) Aug 27 02:31:50.429490 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi Aug 27 02:31:50.429490 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 27 02:31:50.429490 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi Aug 27 02:31:50.434913 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 27 02:31:50.434913 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 120, in show Aug 27 02:31:50.434913 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi vol = self.volume_api.get(context, id) Aug 27 02:31:50.434913 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Aug 27 02:31:50.434913 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Aug 27 02:31:50.434913 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 435, in wrapper Aug 27 02:31:50.434913 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi _reraise(exception.VolumeNotFound(volume_id=volume_id)) Aug 27 02:31:50.434913 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise Aug 27 02:31:50.434913 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) Aug 27 02:31:50.434913 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Aug 27 02:31:50.434913 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Aug 27 02:31:50.434913 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 502, in get Aug 27 02:31:50.434913 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi item = cinderclient( Aug 27 02:31:50.434913 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 202, in get Aug 27 02:31:50.434913 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi return self._get("/volumes/%s" % volume_id, "volume") Aug 27 02:31:50.434913 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 297, in _get Aug 27 02:31:50.434913 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.get(url) Aug 27 02:31:50.434913 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 224, in get Aug 27 02:31:50.434913 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'GET', **kwargs) Aug 27 02:31:50.437479 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Aug 27 02:31:50.437479 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Aug 27 02:31:50.437479 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Aug 27 02:31:50.437479 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Aug 27 02:31:50.437479 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi nova.exception.VolumeNotFound: Volume 0440b424-9658-446b-994e-4e0b2de524d7 could not be found. Aug 27 02:31:50.437479 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi Aug 27 02:31:50.437479 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 27 02:31:50.437479 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi Aug 27 02:31:50.437479 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 27 02:31:50.437479 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 27 02:31:50.437479 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 27 02:31:50.437479 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Aug 27 02:31:50.437479 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Aug 27 02:31:50.437479 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Aug 27 02:31:50.437479 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Aug 27 02:31:50.437479 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Aug 27 02:31:50.437479 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Aug 27 02:31:50.437479 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 122, in show Aug 27 02:31:50.437479 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Aug 27 02:31:50.439087 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Volume 0440b424-9658-446b-994e-4e0b2de524d7 could not be found. Aug 27 02:31:50.439087 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi Aug 27 02:31:50.439087 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 27 02:31:50.439087 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi Aug 27 02:31:50.439087 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 27 02:31:50.439087 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 27 02:31:50.439087 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 27 02:31:50.439087 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Aug 27 02:31:50.439087 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Aug 27 02:31:50.439087 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Volume 0440b424-9658-446b-994e-4e0b2de524d7 could not be found. Aug 27 02:31:50.439087 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi Aug 27 02:31:50.439087 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-f82f55f0-f79e-409a-bdfc-78c0efa2ba72 tempest-VolumesGetTestJSON-514999675 tempest-VolumesGetTestJSON-514999675-project] Returning 404 to user: Volume 0440b424-9658-446b-994e-4e0b2de524d7 could not be found. {{(pid=120864) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Aug 27 02:31:50.439087 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-f82f55f0-f79e-409a-bdfc-78c0efa2ba72 tempest-VolumesGetTestJSON-514999675 tempest-VolumesGetTestJSON-514999675-project] 10.222.0.52 "GET /compute/v2.1/os-volumes/0440b424-9658-446b-994e-4e0b2de524d7" status: 404 len: 109 microversion: 2.1 time: 0.036627 Aug 27 02:31:50.439087 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 439/872] 10.222.0.52 () {58 vars in 1245 bytes} [Sat Aug 27 02:31:50 2022] GET /compute/v2.1/os-volumes/0440b424-9658-446b-994e-4e0b2de524d7 => generated 109 bytes in 38 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Aug 27 02:31:53.376764 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG dbcounter [-] [120863] Writing DB stats nova_api:SELECT=29,nova_api:UPDATE=2 {{(pid=120863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 27 02:31:53.377545 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG dbcounter [-] [120863] Writing DB stats nova_cell1:SELECT=64,nova_cell1:UPDATE=2,nova_cell1:INSERT=2 {{(pid=120863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 27 02:31:53.382483 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-fc08a90c-19e1-498c-9673-9f795753105d tempest-VolumesTestJSON-2040149262 tempest-VolumesTestJSON-2040149262-project] Calling method '.version_select at 0x7f6b1b680e50>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:31:53.436001 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-fc08a90c-19e1-498c-9673-9f795753105d tempest-VolumesTestJSON-2040149262 tempest-VolumesTestJSON-2040149262-project] 10.222.0.52 "GET /compute/v2.1/os-volumes/7afe126b-7908-477b-aff0-cfb0fdcd77a3" status: 200 len: 351 microversion: 2.1 time: 0.060798 Aug 27 02:31:53.436531 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 434/873] 10.222.0.52 () {58 vars in 1245 bytes} [Sat Aug 27 02:31:53 2022] GET /compute/v2.1/os-volumes/7afe126b-7908-477b-aff0-cfb0fdcd77a3 => generated 351 bytes in 62 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 27 02:31:53.654918 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG dbcounter [-] [120864] Writing DB stats nova_api:SELECT=16,nova_api:DELETE=1 {{(pid=120864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 27 02:31:53.655668 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG dbcounter [-] [120864] Writing DB stats nova_cell1:SELECT=30 {{(pid=120864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 27 02:31:53.660104 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-30a1920f-20a0-4f7e-9cd8-079c8879d836 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Calling method '>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:31:53.660846 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-30a1920f-20a0-4f7e-9cd8-079c8879d836 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] [instance: 2ad7bd6d-558b-4b12-a22f-67f33975e5fd] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:31:53.670345 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-30a1920f-20a0-4f7e-9cd8-079c8879d836 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:31:53.670779 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-30a1920f-20a0-4f7e-9cd8-079c8879d836 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:31:53.671244 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-30a1920f-20a0-4f7e-9cd8-079c8879d836 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:31:53.671954 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-30a1920f-20a0-4f7e-9cd8-079c8879d836 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:31:53.672404 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-30a1920f-20a0-4f7e-9cd8-079c8879d836 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:31:53.672820 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-30a1920f-20a0-4f7e-9cd8-079c8879d836 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:31:53.702581 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-30a1920f-20a0-4f7e-9cd8-079c8879d836 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] 10.222.0.52 "GET /compute/v2.1/servers/2ad7bd6d-558b-4b12-a22f-67f33975e5fd/ips" status: 200 len: 150 microversion: 2.1 time: 0.050221 Aug 27 02:31:53.709457 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 440/874] 10.222.0.52 () {58 vars in 1248 bytes} [Sat Aug 27 02:31:53 2022] GET /compute/v2.1/servers/2ad7bd6d-558b-4b12-a22f-67f33975e5fd/ips => generated 150 bytes in 52 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 27 02:31:54.824130 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-96748695-ab5a-4061-82e9-e6ad3021894e tempest-VolumesTestJSON-2040149262 tempest-VolumesTestJSON-2040149262-project] Calling method '.version_select at 0x7f6b1b0e2670>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:31:54.875313 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-96748695-ab5a-4061-82e9-e6ad3021894e tempest-VolumesTestJSON-2040149262 tempest-VolumesTestJSON-2040149262-project] 10.222.0.52 "GET /compute/v2.1/os-volumes/3a88d4cb-5587-45ab-8d1a-600c516bf13a" status: 200 len: 351 microversion: 2.1 time: 0.055209 Aug 27 02:31:54.876088 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 435/875] 10.222.0.52 () {58 vars in 1245 bytes} [Sat Aug 27 02:31:54 2022] GET /compute/v2.1/os-volumes/3a88d4cb-5587-45ab-8d1a-600c516bf13a => generated 351 bytes in 57 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 27 02:31:56.115903 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-4586b334-1b05-4197-a88a-0fce650abc0c tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "f7ac9e7c-7b1e-4f5c-8aa2-3b3858a231eb"}} {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 27 02:31:56.118376 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-4586b334-1b05-4197-a88a-0fce650abc0c tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] [instance: 2ad7bd6d-558b-4b12-a22f-67f33975e5fd] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:31:56.125029 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-4586b334-1b05-4197-a88a-0fce650abc0c tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:31:56.125549 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-4586b334-1b05-4197-a88a-0fce650abc0c tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:31:56.125983 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-4586b334-1b05-4197-a88a-0fce650abc0c tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:31:56.126674 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-4586b334-1b05-4197-a88a-0fce650abc0c tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:31:56.127136 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-4586b334-1b05-4197-a88a-0fce650abc0c tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:31:56.127548 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-4586b334-1b05-4197-a88a-0fce650abc0c tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:31:56.261998 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-be7ddc22-c105-449c-a076-96516e71097a tempest-VolumesTestJSON-2040149262 tempest-VolumesTestJSON-2040149262-project] Calling method '.version_select at 0x7f6b1b035820>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:31:56.324927 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-be7ddc22-c105-449c-a076-96516e71097a tempest-VolumesTestJSON-2040149262 tempest-VolumesTestJSON-2040149262-project] 10.222.0.52 "GET /compute/v2.1/os-volumes/edc64d6f-16b4-41f9-9a13-27c8baa62cd1" status: 200 len: 350 microversion: 2.1 time: 0.067372 Aug 27 02:31:56.325564 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 436/876] 10.222.0.52 () {58 vars in 1245 bytes} [Sat Aug 27 02:31:56 2022] GET /compute/v2.1/os-volumes/edc64d6f-16b4-41f9-9a13-27c8baa62cd1 => generated 350 bytes in 69 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 27 02:31:56.338095 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-fc62b0a1-bfe3-4bf9-90c2-2f2c5b3c28f6 tempest-VolumesTestJSON-2040149262 tempest-VolumesTestJSON-2040149262-project] Calling method '.version_select at 0x7f6b1b2b83a0>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:31:56.421329 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-4586b334-1b05-4197-a88a-0fce650abc0c tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] 10.222.0.52 "POST /compute/v2.1/servers/2ad7bd6d-558b-4b12-a22f-67f33975e5fd/os-volume_attachments" status: 200 len: 194 microversion: 2.1 time: 0.309500 Aug 27 02:31:56.422032 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 441/877] 10.222.0.52 () {60 vars in 1323 bytes} [Sat Aug 27 02:31:56 2022] POST /compute/v2.1/servers/2ad7bd6d-558b-4b12-a22f-67f33975e5fd/os-volume_attachments => generated 194 bytes in 311 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 27 02:31:56.430403 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-fc62b0a1-bfe3-4bf9-90c2-2f2c5b3c28f6 tempest-VolumesTestJSON-2040149262 tempest-VolumesTestJSON-2040149262-project] 10.222.0.52 "GET /compute/v2.1/os-volumes" status: 200 len: 1035 microversion: 2.1 time: 0.095922 Aug 27 02:31:56.431418 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 437/878] 10.222.0.52 () {58 vars in 1134 bytes} [Sat Aug 27 02:31:56 2022] GET /compute/v2.1/os-volumes => generated 1035 bytes in 98 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:31:56.447217 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-500ab5f5-d7a1-4948-bb39-3954af829de0 tempest-VolumesTestJSON-2040149262 tempest-VolumesTestJSON-2040149262-project] Calling method '.version_select at 0x7f6b1e953940>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:31:56.531791 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-500ab5f5-d7a1-4948-bb39-3954af829de0 tempest-VolumesTestJSON-2040149262 tempest-VolumesTestJSON-2040149262-project] 10.222.0.52 "GET /compute/v2.1/os-volumes?limit=2" status: 200 len: 694 microversion: 2.1 time: 0.089476 Aug 27 02:31:56.532433 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 442/879] 10.222.0.52 () {58 vars in 1149 bytes} [Sat Aug 27 02:31:56 2022] GET /compute/v2.1/os-volumes?limit=2 => generated 694 bytes in 91 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 27 02:31:56.546176 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-a4d82afe-cc59-492a-9dc7-8436acc8489a tempest-VolumesTestJSON-2040149262 tempest-VolumesTestJSON-2040149262-project] Calling method '.version_select at 0x7f6b1b0e24c0>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:31:56.632040 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-a4d82afe-cc59-492a-9dc7-8436acc8489a tempest-VolumesTestJSON-2040149262 tempest-VolumesTestJSON-2040149262-project] 10.222.0.52 "GET /compute/v2.1/os-volumes" status: 200 len: 1035 microversion: 2.1 time: 0.089639 Aug 27 02:31:56.632798 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 438/880] 10.222.0.52 () {58 vars in 1134 bytes} [Sat Aug 27 02:31:56 2022] GET /compute/v2.1/os-volumes => generated 1035 bytes in 91 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:31:56.646813 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-145cdc09-4297-4c31-a51f-d05d0dcccef3 tempest-VolumesTestJSON-2040149262 tempest-VolumesTestJSON-2040149262-project] Calling method '.version_select at 0x7f6b1e8c8c10>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:31:56.737210 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-145cdc09-4297-4c31-a51f-d05d0dcccef3 tempest-VolumesTestJSON-2040149262 tempest-VolumesTestJSON-2040149262-project] 10.222.0.52 "GET /compute/v2.1/os-volumes?offset=1&limit=1" status: 200 len: 354 microversion: 2.1 time: 0.094734 Aug 27 02:31:56.737792 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 443/881] 10.222.0.52 () {58 vars in 1167 bytes} [Sat Aug 27 02:31:56 2022] GET /compute/v2.1/os-volumes?offset=1&limit=1 => generated 354 bytes in 96 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 27 02:31:56.751527 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-22829582-5c3f-42c3-89db-18bb34ddbbed tempest-VolumesTestJSON-2040149262 tempest-VolumesTestJSON-2040149262-project] Calling method '.version_select at 0x7f6b1b008f70>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:31:56.856353 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-22829582-5c3f-42c3-89db-18bb34ddbbed tempest-VolumesTestJSON-2040149262 tempest-VolumesTestJSON-2040149262-project] 10.222.0.52 "GET /compute/v2.1/os-volumes/detail?limit=2" status: 200 len: 694 microversion: 2.1 time: 0.108548 Aug 27 02:31:56.856934 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 439/882] 10.222.0.52 () {58 vars in 1170 bytes} [Sat Aug 27 02:31:56 2022] GET /compute/v2.1/os-volumes/detail?limit=2 => generated 694 bytes in 110 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 27 02:31:56.872528 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-0e88564f-7dae-43de-8b00-b2a058d496ae tempest-VolumesTestJSON-2040149262 tempest-VolumesTestJSON-2040149262-project] Calling method '.version_select at 0x7f6b1e8c8d30>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:31:56.982364 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-0e88564f-7dae-43de-8b00-b2a058d496ae tempest-VolumesTestJSON-2040149262 tempest-VolumesTestJSON-2040149262-project] 10.222.0.52 "GET /compute/v2.1/os-volumes/detail" status: 200 len: 1035 microversion: 2.1 time: 0.115111 Aug 27 02:31:56.983123 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 444/883] 10.222.0.52 () {58 vars in 1155 bytes} [Sat Aug 27 02:31:56 2022] GET /compute/v2.1/os-volumes/detail => generated 1035 bytes in 117 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:31:57.001539 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-6be3d6c5-0697-4453-b35e-a7aba008fe8c tempest-VolumesTestJSON-2040149262 tempest-VolumesTestJSON-2040149262-project] Calling method '.version_select at 0x7f6b1b0f1310>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:31:57.095490 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-6be3d6c5-0697-4453-b35e-a7aba008fe8c tempest-VolumesTestJSON-2040149262 tempest-VolumesTestJSON-2040149262-project] 10.222.0.52 "GET /compute/v2.1/os-volumes/detail?offset=1&limit=1" status: 200 len: 354 microversion: 2.1 time: 0.097970 Aug 27 02:31:57.096054 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 440/884] 10.222.0.52 () {58 vars in 1188 bytes} [Sat Aug 27 02:31:56 2022] GET /compute/v2.1/os-volumes/detail?offset=1&limit=1 => generated 354 bytes in 100 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 27 02:31:57.109794 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-bf6902db-516c-4fe2-b213-22971f574ea5 tempest-VolumesTestJSON-2040149262 tempest-VolumesTestJSON-2040149262-project] Calling method '.version_select at 0x7f6b1e8ca430>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:31:57.203104 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-bf6902db-516c-4fe2-b213-22971f574ea5 tempest-VolumesTestJSON-2040149262 tempest-VolumesTestJSON-2040149262-project] 10.222.0.52 "GET /compute/v2.1/os-volumes/detail" status: 200 len: 1035 microversion: 2.1 time: 0.097641 Aug 27 02:31:57.203892 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 445/885] 10.222.0.52 () {58 vars in 1155 bytes} [Sat Aug 27 02:31:57 2022] GET /compute/v2.1/os-volumes/detail => generated 1035 bytes in 99 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:32:00.756844 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [req-8959c823-bfed-4ce7-ad3c-b061ed3070a0 req-d06f032b-fa69-487a-be2f-5f3eb0cb1a13 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "9941c49f-8626-461d-a570-c0ffbfd4b804", "tag": "89f4d982-1807-4e80-b9ef-c7bc0025f1b0"}]} {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 27 02:32:00.761768 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-db85136c-a969-4c10-86d2-d549cbc0ed16 tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:32:00.762485 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-db85136c-a969-4c10-86d2-d549cbc0ed16 tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] [instance: 9941c49f-8626-461d-a570-c0ffbfd4b804] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:32:00.769855 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-db85136c-a969-4c10-86d2-d549cbc0ed16 tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:32:00.771096 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-db85136c-a969-4c10-86d2-d549cbc0ed16 tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:32:00.771762 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-db85136c-a969-4c10-86d2-d549cbc0ed16 tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:32:00.772839 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-db85136c-a969-4c10-86d2-d549cbc0ed16 tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:32:00.773425 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-db85136c-a969-4c10-86d2-d549cbc0ed16 tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:32:00.774943 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-db85136c-a969-4c10-86d2-d549cbc0ed16 tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:32:00.775067 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [req-8959c823-bfed-4ce7-ad3c-b061ed3070a0 req-d06f032b-fa69-487a-be2f-5f3eb0cb1a13 service nova] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:32:00.775465 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [req-8959c823-bfed-4ce7-ad3c-b061ed3070a0 req-d06f032b-fa69-487a-be2f-5f3eb0cb1a13 service nova] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:32:00.775885 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [req-8959c823-bfed-4ce7-ad3c-b061ed3070a0 req-d06f032b-fa69-487a-be2f-5f3eb0cb1a13 service nova] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:32:00.790136 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.compute.server_external_events [req-8959c823-bfed-4ce7-ad3c-b061ed3070a0 req-d06f032b-fa69-487a-be2f-5f3eb0cb1a13 service nova] Creating event network-changed:89f4d982-1807-4e80-b9ef-c7bc0025f1b0 for instance 9941c49f-8626-461d-a570-c0ffbfd4b804 on hv-cinder-84630 Aug 27 02:32:00.797049 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [req-8959c823-bfed-4ce7-ad3c-b061ed3070a0 req-d06f032b-fa69-487a-be2f-5f3eb0cb1a13 service nova] 10.222.0.52 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.042864 Aug 27 02:32:00.801279 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 446/886] 10.222.0.52 () {64 vars in 1317 bytes} [Sat Aug 27 02:32:00 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Aug 27 02:32:00.816417 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-db85136c-a969-4c10-86d2-d549cbc0ed16 tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] [instance: 9941c49f-8626-461d-a570-c0ffbfd4b804] Going to try to terminate instance {{(pid=120863) delete /opt/stack/nova/nova/compute/api.py:2719}} Aug 27 02:32:00.927343 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-db85136c-a969-4c10-86d2-d549cbc0ed16 tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] 10.222.0.52 "DELETE /compute/v2.1/servers/9941c49f-8626-461d-a570-c0ffbfd4b804" status: 204 len: 0 microversion: 2.1 time: 0.176015 Aug 27 02:32:00.929168 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 441/887] 10.222.0.52 () {58 vars in 1239 bytes} [Sat Aug 27 02:32:00 2022] DELETE /compute/v2.1/servers/9941c49f-8626-461d-a570-c0ffbfd4b804 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Aug 27 02:32:00.938794 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-37a3adde-ec04-4794-89b3-0e6115300130 tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] Calling method '>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:32:00.939485 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-37a3adde-ec04-4794-89b3-0e6115300130 tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] [instance: 9941c49f-8626-461d-a570-c0ffbfd4b804] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:32:00.946507 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-37a3adde-ec04-4794-89b3-0e6115300130 tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:32:00.947097 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-37a3adde-ec04-4794-89b3-0e6115300130 tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:32:00.947528 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-37a3adde-ec04-4794-89b3-0e6115300130 tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:32:00.948326 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-37a3adde-ec04-4794-89b3-0e6115300130 tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:32:00.948773 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-37a3adde-ec04-4794-89b3-0e6115300130 tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:32:00.949298 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-37a3adde-ec04-4794-89b3-0e6115300130 tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:32:01.115760 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-37a3adde-ec04-4794-89b3-0e6115300130 tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '642291c4daa04a6ab3fbce92a8be39c6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b2ddff853b7546788b645a0344c2297b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:32:01.124148 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-37a3adde-ec04-4794-89b3-0e6115300130 tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] 10.222.0.52 "GET /compute/v2.1/servers/9941c49f-8626-461d-a570-c0ffbfd4b804" status: 200 len: 1569 microversion: 2.1 time: 0.190232 Aug 27 02:32:01.137250 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 447/888] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:32:00 2022] GET /compute/v2.1/servers/9941c49f-8626-461d-a570-c0ffbfd4b804 => generated 1569 bytes in 203 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:32:02.147927 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-e324ec24-159e-409b-9243-d277ffddfafd tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:32:02.148617 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-e324ec24-159e-409b-9243-d277ffddfafd tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] [instance: 9941c49f-8626-461d-a570-c0ffbfd4b804] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:32:02.156522 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-e324ec24-159e-409b-9243-d277ffddfafd tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:32:02.157006 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-e324ec24-159e-409b-9243-d277ffddfafd tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:32:02.157467 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-e324ec24-159e-409b-9243-d277ffddfafd tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:32:02.158239 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-e324ec24-159e-409b-9243-d277ffddfafd tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:32:02.158708 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-e324ec24-159e-409b-9243-d277ffddfafd tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:32:02.159302 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-e324ec24-159e-409b-9243-d277ffddfafd tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:32:02.304379 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-e324ec24-159e-409b-9243-d277ffddfafd tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '642291c4daa04a6ab3fbce92a8be39c6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b2ddff853b7546788b645a0344c2297b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:32:02.313314 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-e324ec24-159e-409b-9243-d277ffddfafd tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] 10.222.0.52 "GET /compute/v2.1/servers/9941c49f-8626-461d-a570-c0ffbfd4b804" status: 200 len: 1569 microversion: 2.1 time: 0.169619 Aug 27 02:32:02.314156 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 442/889] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:32:02 2022] GET /compute/v2.1/servers/9941c49f-8626-461d-a570-c0ffbfd4b804 => generated 1569 bytes in 172 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 27 02:32:03.334794 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-3d393fe8-bbc0-4891-911d-4637f06c905e tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] Calling method '>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:32:03.335374 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-3d393fe8-bbc0-4891-911d-4637f06c905e tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] [instance: 9941c49f-8626-461d-a570-c0ffbfd4b804] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:32:03.343578 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-3d393fe8-bbc0-4891-911d-4637f06c905e tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:32:03.344062 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-3d393fe8-bbc0-4891-911d-4637f06c905e tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:32:03.344495 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-3d393fe8-bbc0-4891-911d-4637f06c905e tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:32:03.345233 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-3d393fe8-bbc0-4891-911d-4637f06c905e tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:32:03.345684 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-3d393fe8-bbc0-4891-911d-4637f06c905e tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:32:03.346104 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-3d393fe8-bbc0-4891-911d-4637f06c905e tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:32:03.537172 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-3d393fe8-bbc0-4891-911d-4637f06c905e tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '642291c4daa04a6ab3fbce92a8be39c6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b2ddff853b7546788b645a0344c2297b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:32:03.547230 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-3d393fe8-bbc0-4891-911d-4637f06c905e tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] 10.222.0.52 "GET /compute/v2.1/servers/9941c49f-8626-461d-a570-c0ffbfd4b804" status: 200 len: 1450 microversion: 2.1 time: 0.216191 Aug 27 02:32:03.547799 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 448/890] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:32:03 2022] GET /compute/v2.1/servers/9941c49f-8626-461d-a570-c0ffbfd4b804 => generated 1450 bytes in 217 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:32:03.967809 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [req-24ca0e15-d69a-4e9b-af35-00617ce5efeb req-0466011a-3ac5-4e85-9e73-5e299de81b1e service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "9941c49f-8626-461d-a570-c0ffbfd4b804", "name": "network-vif-deleted", "tag": "89f4d982-1807-4e80-b9ef-c7bc0025f1b0"}]} {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 27 02:32:03.982866 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [req-24ca0e15-d69a-4e9b-af35-00617ce5efeb req-0466011a-3ac5-4e85-9e73-5e299de81b1e service nova] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:32:03.983331 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [req-24ca0e15-d69a-4e9b-af35-00617ce5efeb req-0466011a-3ac5-4e85-9e73-5e299de81b1e service nova] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:32:03.983929 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [req-24ca0e15-d69a-4e9b-af35-00617ce5efeb req-0466011a-3ac5-4e85-9e73-5e299de81b1e service nova] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:32:03.999265 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.compute.server_external_events [req-24ca0e15-d69a-4e9b-af35-00617ce5efeb req-0466011a-3ac5-4e85-9e73-5e299de81b1e service nova] Creating event network-vif-deleted:89f4d982-1807-4e80-b9ef-c7bc0025f1b0 for instance 9941c49f-8626-461d-a570-c0ffbfd4b804 on hv-cinder-84630 Aug 27 02:32:04.006326 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [req-24ca0e15-d69a-4e9b-af35-00617ce5efeb req-0466011a-3ac5-4e85-9e73-5e299de81b1e service nova] 10.222.0.52 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.043516 Aug 27 02:32:04.006895 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 443/891] 10.222.0.52 () {64 vars in 1317 bytes} [Sat Aug 27 02:32:03 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 27 02:32:04.035435 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-79f4f636-af33-4ca3-85e2-6a6653d86ec6 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "f7ac9e7c-7b1e-4f5c-8aa2-3b3858a231eb"}} {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 27 02:32:04.038903 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-79f4f636-af33-4ca3-85e2-6a6653d86ec6 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] [instance: 2ad7bd6d-558b-4b12-a22f-67f33975e5fd] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:32:04.048636 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-79f4f636-af33-4ca3-85e2-6a6653d86ec6 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:32:04.049138 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-79f4f636-af33-4ca3-85e2-6a6653d86ec6 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:32:04.049653 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-79f4f636-af33-4ca3-85e2-6a6653d86ec6 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:32:04.050413 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-79f4f636-af33-4ca3-85e2-6a6653d86ec6 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:32:04.051083 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-79f4f636-af33-4ca3-85e2-6a6653d86ec6 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:32:04.051589 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-79f4f636-af33-4ca3-85e2-6a6653d86ec6 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:32:04.352760 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.wsgi [None req-79f4f636-af33-4ca3-85e2-6a6653d86ec6 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] HTTP exception thrown: Invalid volume: volume f7ac9e7c-7b1e-4f5c-8aa2-3b3858a231eb is already attached to instances: 2ad7bd6d-558b-4b12-a22f-67f33975e5fd Aug 27 02:32:04.354849 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-79f4f636-af33-4ca3-85e2-6a6653d86ec6 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Request method failure captured: Aug 27 02:32:04.354849 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: request: POST /compute/v2.1/servers/2ad7bd6d-558b-4b12-a22f-67f33975e5fd/os-volume_attachments HTTP/1.1 Aug 27 02:32:04.354849 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: Accept: application/json Aug 27 02:32:04.354849 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: Accept-Encoding: identity Aug 27 02:32:04.354849 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: Connection: close Aug 27 02:32:04.354849 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: Content-Length: 74 Aug 27 02:32:04.354849 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: Content-Type: application/json Aug 27 02:32:04.354849 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: Host: 10.222.0.52 Aug 27 02:32:04.354849 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: Openstack-System-Scope: None Aug 27 02:32:04.354849 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: User-Agent: python-urllib3/1.26.11 Aug 27 02:32:04.354849 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-Auth-Token: gAAAAABjCX_pkOb4-Fffq6Hbe0cfjlj1ARL1JmORSzrYWucWmWgZ67LEorDD3IWpPkj3UP534US8GW_nja0iAKwcdxSx3e5teQzZMdrCvOvA5aSGJO8iodOjhfOIsl8L0d8om8kj8_3q-8uTHxKupSTnIQy_J-f1kC4e22VNejIIiExDsGitats Aug 27 02:32:04.354849 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-Domain-Id: None Aug 27 02:32:04.354849 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-Domain-Name: None Aug 27 02:32:04.354849 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-Identity-Status: Confirmed Aug 27 02:32:04.354849 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-Is-Admin-Project: True Aug 27 02:32:04.354849 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-Project-Domain-Id: default Aug 27 02:32:04.354849 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-Project-Domain-Name: Default Aug 27 02:32:04.354849 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-Project-Id: 5d5ec8450c0941d18d63558baab54737 Aug 27 02:32:04.354849 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-Project-Name: tempest-AttachVolumeNegativeTest-1988669364 Aug 27 02:32:04.354849 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-Role: reader,member Aug 27 02:32:04.354849 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-Roles: reader,member Aug 27 02:32:04.356527 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-Service-Catalog: [{"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.52:8080", "publicURL": "http://10.222.0.52:8080/v1/AUTH_5d5ec8450c0941d18d63558baab54737"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52:9696/networking"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/compute/v2/5d5ec8450c0941d18d63558baab54737"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/image"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/volume/v3/5d5ec8450c0941d18d63558baab54737"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/identity"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/volume/v3/5d5ec8450c0941d18d63558baab54737"}]}] Aug 27 02:32:04.356527 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-Tenant: tempest-AttachVolumeNegativeTest-1988669364 Aug 27 02:32:04.356527 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-Tenant-Id: 5d5ec8450c0941d18d63558baab54737 Aug 27 02:32:04.356527 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-1988669364 Aug 27 02:32:04.356527 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-User: tempest-AttachVolumeNegativeTest-1988669364-project Aug 27 02:32:04.356527 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-User-Domain-Id: default Aug 27 02:32:04.356527 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-User-Domain-Name: Default Aug 27 02:32:04.356527 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-User-Id: 26a86351583d49d090b3db09f127d49f Aug 27 02:32:04.356527 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-User-Name: tempest-AttachVolumeNegativeTest-1988669364-project Aug 27 02:32:04.356527 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: Aug 27 02:32:04.356527 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: {"volumeAttachment": {"volumeId": "f7ac9e7c-7b1e-4f5c-8aa2-3b3858a231eb"}} Aug 27 02:32:04.356527 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: method: > Aug 27 02:32:04.357487 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: exception: Invalid volume: volume f7ac9e7c-7b1e-4f5c-8aa2-3b3858a231eb is already attached to instances: 2ad7bd6d-558b-4b12-a22f-67f33975e5fd Aug 27 02:32:04.357487 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: action_args: {'server_id': '2ad7bd6d-558b-4b12-a22f-67f33975e5fd', 'body': {'volumeAttachment': {'volumeId': 'f7ac9e7c-7b1e-4f5c-8aa2-3b3858a231eb'}}} Aug 27 02:32:04.357487 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Aug 27 02:32:04.357487 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 27 02:32:04.357487 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 388, in create Aug 27 02:32:04.357487 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi device = self.compute_api.attach_volume( Aug 27 02:32:04.357487 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 241, in inner Aug 27 02:32:04.357487 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Aug 27 02:32:04.357487 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 167, in inner Aug 27 02:32:04.357487 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi return f(self, context, instance, *args, **kw) Aug 27 02:32:04.357487 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5128, in attach_volume Aug 27 02:32:04.357487 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi self._check_volume_already_attached(context, instance, volume) Aug 27 02:32:04.357487 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 4968, in _check_volume_already_attached Aug 27 02:32:04.357487 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi raise exception.InvalidVolume(reason=msg) Aug 27 02:32:04.357487 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi nova.exception.InvalidVolume: Invalid volume: volume f7ac9e7c-7b1e-4f5c-8aa2-3b3858a231eb is already attached to instances: 2ad7bd6d-558b-4b12-a22f-67f33975e5fd Aug 27 02:32:04.357487 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi Aug 27 02:32:04.357487 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 27 02:32:04.357487 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi Aug 27 02:32:04.357487 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 27 02:32:04.359774 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 27 02:32:04.359774 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 27 02:32:04.359774 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Aug 27 02:32:04.359774 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Aug 27 02:32:04.359774 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Aug 27 02:32:04.359774 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Aug 27 02:32:04.359774 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Aug 27 02:32:04.359774 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Aug 27 02:32:04.359774 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Aug 27 02:32:04.359774 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Aug 27 02:32:04.359774 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Aug 27 02:32:04.359774 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Aug 27 02:32:04.359774 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 406, in create Aug 27 02:32:04.359774 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi raise exc.HTTPBadRequest(explanation=e.format_message()) Aug 27 02:32:04.359774 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi webob.exc.HTTPBadRequest: Invalid volume: volume f7ac9e7c-7b1e-4f5c-8aa2-3b3858a231eb is already attached to instances: 2ad7bd6d-558b-4b12-a22f-67f33975e5fd Aug 27 02:32:04.359774 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi Aug 27 02:32:04.359774 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 27 02:32:04.359774 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi Aug 27 02:32:04.359774 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 27 02:32:04.359774 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 27 02:32:04.361038 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 27 02:32:04.361038 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Aug 27 02:32:04.361038 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Aug 27 02:32:04.361038 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid volume: volume f7ac9e7c-7b1e-4f5c-8aa2-3b3858a231eb is already attached to instances: 2ad7bd6d-558b-4b12-a22f-67f33975e5fd Aug 27 02:32:04.361038 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi Aug 27 02:32:04.361038 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-79f4f636-af33-4ca3-85e2-6a6653d86ec6 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Returning 400 to user: Invalid volume: volume f7ac9e7c-7b1e-4f5c-8aa2-3b3858a231eb is already attached to instances: 2ad7bd6d-558b-4b12-a22f-67f33975e5fd {{(pid=120864) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Aug 27 02:32:04.361038 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-79f4f636-af33-4ca3-85e2-6a6653d86ec6 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] 10.222.0.52 "POST /compute/v2.1/servers/2ad7bd6d-558b-4b12-a22f-67f33975e5fd/os-volume_attachments" status: 400 len: 174 microversion: 2.1 time: 0.327561 Aug 27 02:32:04.361038 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 449/892] 10.222.0.52 () {60 vars in 1323 bytes} [Sat Aug 27 02:32:04 2022] POST /compute/v2.1/servers/2ad7bd6d-558b-4b12-a22f-67f33975e5fd/os-volume_attachments => generated 174 bytes in 329 msecs (HTTP/1.1 400) 9 headers in 381 bytes (1 switches on core 0) Aug 27 02:32:04.456732 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-cc905c6f-6b40-44ef-9b86-29dcb8d982f3 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:32:04.457422 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-cc905c6f-6b40-44ef-9b86-29dcb8d982f3 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] [instance: 2ad7bd6d-558b-4b12-a22f-67f33975e5fd] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:32:04.464791 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-cc905c6f-6b40-44ef-9b86-29dcb8d982f3 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:32:04.466733 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-cc905c6f-6b40-44ef-9b86-29dcb8d982f3 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:32:04.467846 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-cc905c6f-6b40-44ef-9b86-29dcb8d982f3 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:32:04.468631 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-cc905c6f-6b40-44ef-9b86-29dcb8d982f3 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:32:04.469336 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-cc905c6f-6b40-44ef-9b86-29dcb8d982f3 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:32:04.469949 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-cc905c6f-6b40-44ef-9b86-29dcb8d982f3 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:32:04.555325 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.objects.instance [None req-cc905c6f-6b40-44ef-9b86-29dcb8d982f3 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lazy-loading 'services' on Instance uuid 2ad7bd6d-558b-4b12-a22f-67f33975e5fd {{(pid=120863) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Aug 27 02:32:04.567795 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-30ea3d46-9f69-4dbf-8249-40a9d9b63f04 tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] Calling method '>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:32:04.568423 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-30ea3d46-9f69-4dbf-8249-40a9d9b63f04 tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] [instance: 9941c49f-8626-461d-a570-c0ffbfd4b804] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:32:04.576317 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-30ea3d46-9f69-4dbf-8249-40a9d9b63f04 tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:32:04.576317 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-30ea3d46-9f69-4dbf-8249-40a9d9b63f04 tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:32:04.576639 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-30ea3d46-9f69-4dbf-8249-40a9d9b63f04 tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:32:04.577345 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-30ea3d46-9f69-4dbf-8249-40a9d9b63f04 tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:32:04.577787 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-30ea3d46-9f69-4dbf-8249-40a9d9b63f04 tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:32:04.578199 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-30ea3d46-9f69-4dbf-8249-40a9d9b63f04 tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:32:04.610878 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-30ea3d46-9f69-4dbf-8249-40a9d9b63f04 tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:32:04.611382 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-30ea3d46-9f69-4dbf-8249-40a9d9b63f04 tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:32:04.611790 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-30ea3d46-9f69-4dbf-8249-40a9d9b63f04 tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:32:04.612228 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.objects.instance [None req-30ea3d46-9f69-4dbf-8249-40a9d9b63f04 tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] Lazy-loading 'fault' on Instance uuid 9941c49f-8626-461d-a570-c0ffbfd4b804 {{(pid=120864) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Aug 27 02:32:04.645723 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-cc905c6f-6b40-44ef-9b86-29dcb8d982f3 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] 10.222.0.52 "DELETE /compute/v2.1/servers/2ad7bd6d-558b-4b12-a22f-67f33975e5fd/os-volume_attachments/f7ac9e7c-7b1e-4f5c-8aa2-3b3858a231eb" status: 202 len: 0 microversion: 2.1 time: 0.193011 Aug 27 02:32:04.646394 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 444/893] 10.222.0.52 () {58 vars in 1416 bytes} [Sat Aug 27 02:32:04 2022] DELETE /compute/v2.1/servers/2ad7bd6d-558b-4b12-a22f-67f33975e5fd/os-volume_attachments/f7ac9e7c-7b1e-4f5c-8aa2-3b3858a231eb => generated 0 bytes in 195 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Aug 27 02:32:04.664950 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-1b7cfa4d-1b46-4f93-adad-113658a15cdf tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:32:04.667495 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-1b7cfa4d-1b46-4f93-adad-113658a15cdf tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] [instance: 2ad7bd6d-558b-4b12-a22f-67f33975e5fd] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:32:04.669061 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-30ea3d46-9f69-4dbf-8249-40a9d9b63f04 tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '642291c4daa04a6ab3fbce92a8be39c6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b2ddff853b7546788b645a0344c2297b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:32:04.676215 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-1b7cfa4d-1b46-4f93-adad-113658a15cdf tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:32:04.676794 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-1b7cfa4d-1b46-4f93-adad-113658a15cdf tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:32:04.677401 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-1b7cfa4d-1b46-4f93-adad-113658a15cdf tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:32:04.678225 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-30ea3d46-9f69-4dbf-8249-40a9d9b63f04 tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] 10.222.0.52 "GET /compute/v2.1/servers/9941c49f-8626-461d-a570-c0ffbfd4b804" status: 200 len: 1225 microversion: 2.1 time: 0.114196 Aug 27 02:32:04.678383 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-1b7cfa4d-1b46-4f93-adad-113658a15cdf tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:32:04.678999 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-1b7cfa4d-1b46-4f93-adad-113658a15cdf tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:32:04.679404 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 450/894] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:32:04 2022] GET /compute/v2.1/servers/9941c49f-8626-461d-a570-c0ffbfd4b804 => generated 1225 bytes in 116 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:32:04.679621 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-1b7cfa4d-1b46-4f93-adad-113658a15cdf tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:32:04.720113 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-1b7cfa4d-1b46-4f93-adad-113658a15cdf tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] 10.222.0.52 "GET /compute/v2.1/servers/2ad7bd6d-558b-4b12-a22f-67f33975e5fd/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.061989 Aug 27 02:32:04.720667 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 445/895] 10.222.0.52 () {58 vars in 1302 bytes} [Sat Aug 27 02:32:04 2022] GET /compute/v2.1/servers/2ad7bd6d-558b-4b12-a22f-67f33975e5fd/os-volume_attachments => generated 197 bytes in 71 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 27 02:32:05.703037 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-53807763-f7a6-46df-ab6b-37a57c80f5f4 tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] Calling method '>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:32:05.703647 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-53807763-f7a6-46df-ab6b-37a57c80f5f4 tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] [instance: 9941c49f-8626-461d-a570-c0ffbfd4b804] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:32:05.711256 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-53807763-f7a6-46df-ab6b-37a57c80f5f4 tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:32:05.711737 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-53807763-f7a6-46df-ab6b-37a57c80f5f4 tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:32:05.712162 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-53807763-f7a6-46df-ab6b-37a57c80f5f4 tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:32:05.712878 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-53807763-f7a6-46df-ab6b-37a57c80f5f4 tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:32:05.713362 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-53807763-f7a6-46df-ab6b-37a57c80f5f4 tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:32:05.713777 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-53807763-f7a6-46df-ab6b-37a57c80f5f4 tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:32:05.730184 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.wsgi [None req-53807763-f7a6-46df-ab6b-37a57c80f5f4 tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] HTTP exception thrown: Instance 9941c49f-8626-461d-a570-c0ffbfd4b804 could not be found. Aug 27 02:32:05.731847 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-53807763-f7a6-46df-ab6b-37a57c80f5f4 tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] Request method failure captured: Aug 27 02:32:05.731847 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: request: GET /compute/v2.1/servers/9941c49f-8626-461d-a570-c0ffbfd4b804 HTTP/1.1 Aug 27 02:32:05.731847 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: Accept: application/json Aug 27 02:32:05.731847 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: Accept-Encoding: identity Aug 27 02:32:05.731847 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: Connection: close Aug 27 02:32:05.731847 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: Content-Length: 0 Aug 27 02:32:05.731847 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: Content-Type: application/json Aug 27 02:32:05.731847 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: Host: 10.222.0.52 Aug 27 02:32:05.731847 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: Openstack-System-Scope: None Aug 27 02:32:05.731847 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: User-Agent: python-urllib3/1.26.11 Aug 27 02:32:05.731847 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-Auth-Token: gAAAAABjCYBrbSmb8CNf5lAZdYTN_-PY7ZxRPKRwcC5E8xVy-iIl-b2LKXI8W_3LApeM_YvaKZH5iY6mF5-lSiukhlYoQC9JOj4sbw_CvgOzjJMcOouE7Tkrnea41Q9cvSSJAK-OM8-8so-gtXwpvNF7AzhpQY-1NR8vJWZJ7SyOeHY5x0uBzfY Aug 27 02:32:05.731847 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-Domain-Id: None Aug 27 02:32:05.731847 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-Domain-Name: None Aug 27 02:32:05.731847 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-Identity-Status: Confirmed Aug 27 02:32:05.731847 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-Is-Admin-Project: True Aug 27 02:32:05.731847 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-Project-Domain-Id: default Aug 27 02:32:05.731847 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-Project-Domain-Name: Default Aug 27 02:32:05.731847 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-Project-Id: b2ddff853b7546788b645a0344c2297b Aug 27 02:32:05.731847 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-Project-Name: tempest-TestShelveInstance-885432820 Aug 27 02:32:05.731847 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-Role: reader,member Aug 27 02:32:05.731847 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-Roles: reader,member Aug 27 02:32:05.733392 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-Service-Catalog: [{"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.52:8080", "publicURL": "http://10.222.0.52:8080/v1/AUTH_b2ddff853b7546788b645a0344c2297b"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52:9696/networking"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/compute/v2/b2ddff853b7546788b645a0344c2297b"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/image"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/volume/v3/b2ddff853b7546788b645a0344c2297b"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/identity"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/volume/v3/b2ddff853b7546788b645a0344c2297b"}]}] Aug 27 02:32:05.733392 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-Tenant: tempest-TestShelveInstance-885432820 Aug 27 02:32:05.733392 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-Tenant-Id: b2ddff853b7546788b645a0344c2297b Aug 27 02:32:05.733392 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-Tenant-Name: tempest-TestShelveInstance-885432820 Aug 27 02:32:05.733392 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-User: tempest-TestShelveInstance-885432820-project Aug 27 02:32:05.733392 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-User-Domain-Id: default Aug 27 02:32:05.733392 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-User-Domain-Name: Default Aug 27 02:32:05.733392 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-User-Id: 642291c4daa04a6ab3fbce92a8be39c6 Aug 27 02:32:05.733392 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-User-Name: tempest-TestShelveInstance-885432820-project Aug 27 02:32:05.733392 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: method: > Aug 27 02:32:05.733392 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: exception: Instance 9941c49f-8626-461d-a570-c0ffbfd4b804 could not be found. Aug 27 02:32:05.733392 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: action_args: {'id': '9941c49f-8626-461d-a570-c0ffbfd4b804'} Aug 27 02:32:05.734333 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Aug 27 02:32:05.734333 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 27 02:32:05.734333 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Aug 27 02:32:05.734333 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Aug 27 02:32:05.734333 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Aug 27 02:32:05.734333 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Aug 27 02:32:05.734333 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Aug 27 02:32:05.734333 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Aug 27 02:32:05.734333 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Aug 27 02:32:05.734333 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Aug 27 02:32:05.734333 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 9941c49f-8626-461d-a570-c0ffbfd4b804 could not be found. Aug 27 02:32:05.734333 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi Aug 27 02:32:05.734333 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 27 02:32:05.734333 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi Aug 27 02:32:05.734333 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 27 02:32:05.734333 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 27 02:32:05.734333 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 27 02:32:05.734333 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Aug 27 02:32:05.734333 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Aug 27 02:32:05.734333 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Aug 27 02:32:05.735759 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Aug 27 02:32:05.735759 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Aug 27 02:32:05.735759 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi instance = self._get_server( Aug 27 02:32:05.735759 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Aug 27 02:32:05.735759 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Aug 27 02:32:05.735759 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Aug 27 02:32:05.735759 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Aug 27 02:32:05.735759 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 9941c49f-8626-461d-a570-c0ffbfd4b804 could not be found. Aug 27 02:32:05.735759 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi Aug 27 02:32:05.735759 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 27 02:32:05.735759 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi Aug 27 02:32:05.735759 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 27 02:32:05.735759 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 27 02:32:05.735759 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 27 02:32:05.735759 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Aug 27 02:32:05.735759 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Aug 27 02:32:05.735759 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 9941c49f-8626-461d-a570-c0ffbfd4b804 could not be found. Aug 27 02:32:05.735759 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi Aug 27 02:32:05.737094 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-53807763-f7a6-46df-ab6b-37a57c80f5f4 tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] Returning 404 to user: Instance 9941c49f-8626-461d-a570-c0ffbfd4b804 could not be found. {{(pid=120864) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Aug 27 02:32:05.737199 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-529e4df3-4ccd-4f3e-a8d9-b3f793066bff tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:32:05.737690 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-53807763-f7a6-46df-ab6b-37a57c80f5f4 tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] 10.222.0.52 "GET /compute/v2.1/servers/9941c49f-8626-461d-a570-c0ffbfd4b804" status: 404 len: 111 microversion: 2.1 time: 0.035335 Aug 27 02:32:05.737690 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 451/896] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:32:05 2022] GET /compute/v2.1/servers/9941c49f-8626-461d-a570-c0ffbfd4b804 => generated 111 bytes in 37 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Aug 27 02:32:05.738695 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-529e4df3-4ccd-4f3e-a8d9-b3f793066bff tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] [instance: 2ad7bd6d-558b-4b12-a22f-67f33975e5fd] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:32:05.745466 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-529e4df3-4ccd-4f3e-a8d9-b3f793066bff tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:32:05.745961 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-529e4df3-4ccd-4f3e-a8d9-b3f793066bff tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:32:05.746393 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-529e4df3-4ccd-4f3e-a8d9-b3f793066bff tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:32:05.747284 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-529e4df3-4ccd-4f3e-a8d9-b3f793066bff tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:32:05.747737 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-529e4df3-4ccd-4f3e-a8d9-b3f793066bff tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:32:05.748162 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-529e4df3-4ccd-4f3e-a8d9-b3f793066bff tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:32:05.784435 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-529e4df3-4ccd-4f3e-a8d9-b3f793066bff tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] 10.222.0.52 "GET /compute/v2.1/servers/2ad7bd6d-558b-4b12-a22f-67f33975e5fd/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.055575 Aug 27 02:32:05.785541 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 446/897] 10.222.0.52 () {58 vars in 1302 bytes} [Sat Aug 27 02:32:05 2022] GET /compute/v2.1/servers/2ad7bd6d-558b-4b12-a22f-67f33975e5fd/os-volume_attachments => generated 197 bytes in 57 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 27 02:32:05.922229 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-b1a9d222-626d-42e6-a676-4160a959607e tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] Calling method '.version_select at 0x7f6b1e8ca1f0>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:32:05.956907 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-b1a9d222-626d-42e6-a676-4160a959607e tempest-TestShelveInstance-885432820 tempest-TestShelveInstance-885432820-project] 10.222.0.52 "DELETE /compute/v2.1/os-keypairs/tempest-TestShelveInstance-1152661178" status: 202 len: 0 microversion: 2.1 time: 0.038499 Aug 27 02:32:05.957455 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 452/898] 10.222.0.52 () {58 vars in 1254 bytes} [Sat Aug 27 02:32:05 2022] DELETE /compute/v2.1/os-keypairs/tempest-TestShelveInstance-1152661178 => generated 0 bytes in 40 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Aug 27 02:32:06.798908 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-774abf23-aa0d-44fa-bb84-7932e98c285a tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:32:06.801210 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-774abf23-aa0d-44fa-bb84-7932e98c285a tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] [instance: 2ad7bd6d-558b-4b12-a22f-67f33975e5fd] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:32:06.807834 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-774abf23-aa0d-44fa-bb84-7932e98c285a tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:32:06.808312 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-774abf23-aa0d-44fa-bb84-7932e98c285a tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:32:06.808988 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-774abf23-aa0d-44fa-bb84-7932e98c285a tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:32:06.810167 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-774abf23-aa0d-44fa-bb84-7932e98c285a tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:32:06.810625 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-774abf23-aa0d-44fa-bb84-7932e98c285a tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:32:06.811045 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-774abf23-aa0d-44fa-bb84-7932e98c285a tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:32:06.848241 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-774abf23-aa0d-44fa-bb84-7932e98c285a tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] 10.222.0.52 "GET /compute/v2.1/servers/2ad7bd6d-558b-4b12-a22f-67f33975e5fd/os-volume_attachments" status: 200 len: 25 microversion: 2.1 time: 0.053927 Aug 27 02:32:06.848896 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 447/899] 10.222.0.52 () {58 vars in 1302 bytes} [Sat Aug 27 02:32:06 2022] GET /compute/v2.1/servers/2ad7bd6d-558b-4b12-a22f-67f33975e5fd/os-volume_attachments => generated 25 bytes in 56 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Aug 27 02:32:06.924871 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-296677ac-1331-4188-8018-ec9c235e6e72 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Calling method '>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:32:06.925509 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-296677ac-1331-4188-8018-ec9c235e6e72 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] [instance: 2ad7bd6d-558b-4b12-a22f-67f33975e5fd] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:32:06.933101 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-296677ac-1331-4188-8018-ec9c235e6e72 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:32:06.933850 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-296677ac-1331-4188-8018-ec9c235e6e72 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:32:06.934286 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-296677ac-1331-4188-8018-ec9c235e6e72 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:32:06.934988 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-296677ac-1331-4188-8018-ec9c235e6e72 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:32:06.935441 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-296677ac-1331-4188-8018-ec9c235e6e72 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:32:06.935836 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-296677ac-1331-4188-8018-ec9c235e6e72 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:32:06.961349 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-296677ac-1331-4188-8018-ec9c235e6e72 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] [instance: 2ad7bd6d-558b-4b12-a22f-67f33975e5fd] Going to try to terminate instance {{(pid=120864) delete /opt/stack/nova/nova/compute/api.py:2719}} Aug 27 02:32:07.060217 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-296677ac-1331-4188-8018-ec9c235e6e72 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] 10.222.0.52 "DELETE /compute/v2.1/servers/2ad7bd6d-558b-4b12-a22f-67f33975e5fd" status: 204 len: 0 microversion: 2.1 time: 0.139529 Aug 27 02:32:07.060781 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 453/900] 10.222.0.52 () {58 vars in 1239 bytes} [Sat Aug 27 02:32:06 2022] DELETE /compute/v2.1/servers/2ad7bd6d-558b-4b12-a22f-67f33975e5fd => generated 0 bytes in 141 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Aug 27 02:32:07.069431 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-f0e92fef-4c1d-44f5-83c7-b77ed85f5f6d tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:32:07.070051 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-f0e92fef-4c1d-44f5-83c7-b77ed85f5f6d tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] [instance: 2ad7bd6d-558b-4b12-a22f-67f33975e5fd] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:32:07.078154 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-f0e92fef-4c1d-44f5-83c7-b77ed85f5f6d tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:32:07.079318 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-f0e92fef-4c1d-44f5-83c7-b77ed85f5f6d tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:32:07.079694 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-f0e92fef-4c1d-44f5-83c7-b77ed85f5f6d tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:32:07.080582 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-f0e92fef-4c1d-44f5-83c7-b77ed85f5f6d tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:32:07.081209 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-f0e92fef-4c1d-44f5-83c7-b77ed85f5f6d tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:32:07.081779 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-f0e92fef-4c1d-44f5-83c7-b77ed85f5f6d tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:32:07.318041 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-f0e92fef-4c1d-44f5-83c7-b77ed85f5f6d tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '26a86351583d49d090b3db09f127d49f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5d5ec8450c0941d18d63558baab54737', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:32:07.328792 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-f0e92fef-4c1d-44f5-83c7-b77ed85f5f6d tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] 10.222.0.52 "GET /compute/v2.1/servers/2ad7bd6d-558b-4b12-a22f-67f33975e5fd" status: 200 len: 1681 microversion: 2.1 time: 0.263385 Aug 27 02:32:07.329332 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 448/901] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:32:07 2022] GET /compute/v2.1/servers/2ad7bd6d-558b-4b12-a22f-67f33975e5fd => generated 1681 bytes in 265 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 27 02:32:08.352054 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-f04a87a8-c9f8-4f09-9b12-6dc0f8594bfb tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Calling method '>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:32:08.352755 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-f04a87a8-c9f8-4f09-9b12-6dc0f8594bfb tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] [instance: 2ad7bd6d-558b-4b12-a22f-67f33975e5fd] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:32:08.359823 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-f04a87a8-c9f8-4f09-9b12-6dc0f8594bfb tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:32:08.360534 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-f04a87a8-c9f8-4f09-9b12-6dc0f8594bfb tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:32:08.361581 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-f04a87a8-c9f8-4f09-9b12-6dc0f8594bfb tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:32:08.362529 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-f04a87a8-c9f8-4f09-9b12-6dc0f8594bfb tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:32:08.363156 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-f04a87a8-c9f8-4f09-9b12-6dc0f8594bfb tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:32:08.363574 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-f04a87a8-c9f8-4f09-9b12-6dc0f8594bfb tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:32:08.866009 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-f04a87a8-c9f8-4f09-9b12-6dc0f8594bfb tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '26a86351583d49d090b3db09f127d49f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5d5ec8450c0941d18d63558baab54737', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:32:08.874753 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-f04a87a8-c9f8-4f09-9b12-6dc0f8594bfb tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] 10.222.0.52 "GET /compute/v2.1/servers/2ad7bd6d-558b-4b12-a22f-67f33975e5fd" status: 200 len: 1681 microversion: 2.1 time: 0.527130 Aug 27 02:32:08.875330 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 454/902] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:32:08 2022] GET /compute/v2.1/servers/2ad7bd6d-558b-4b12-a22f-67f33975e5fd => generated 1681 bytes in 529 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:32:09.897144 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-5819679b-db01-4eaf-95d1-12e6c3163f78 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:32:09.898268 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-5819679b-db01-4eaf-95d1-12e6c3163f78 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] [instance: 2ad7bd6d-558b-4b12-a22f-67f33975e5fd] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:32:09.908376 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-5819679b-db01-4eaf-95d1-12e6c3163f78 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:32:09.909024 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-5819679b-db01-4eaf-95d1-12e6c3163f78 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:32:09.909616 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-5819679b-db01-4eaf-95d1-12e6c3163f78 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:32:09.910431 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-5819679b-db01-4eaf-95d1-12e6c3163f78 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:32:09.910979 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-5819679b-db01-4eaf-95d1-12e6c3163f78 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:32:09.911547 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-5819679b-db01-4eaf-95d1-12e6c3163f78 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:32:10.232125 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-5819679b-db01-4eaf-95d1-12e6c3163f78 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '26a86351583d49d090b3db09f127d49f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5d5ec8450c0941d18d63558baab54737', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:32:10.241819 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-5819679b-db01-4eaf-95d1-12e6c3163f78 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] 10.222.0.52 "GET /compute/v2.1/servers/2ad7bd6d-558b-4b12-a22f-67f33975e5fd" status: 200 len: 1681 microversion: 2.1 time: 0.349258 Aug 27 02:32:10.243032 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 449/903] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:32:09 2022] GET /compute/v2.1/servers/2ad7bd6d-558b-4b12-a22f-67f33975e5fd => generated 1681 bytes in 352 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:32:10.748683 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [req-bc6ce08c-7f0f-463e-884c-67c8ea03ba9f req-8cd75f4e-3844-4e67-a480-dc6e6952c536 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "2ad7bd6d-558b-4b12-a22f-67f33975e5fd", "name": "network-vif-deleted", "tag": "d49ef841-e8bf-49f7-b85b-a7fbc34a1e23"}]} {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 27 02:32:10.773532 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [req-bc6ce08c-7f0f-463e-884c-67c8ea03ba9f req-8cd75f4e-3844-4e67-a480-dc6e6952c536 service nova] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:32:10.774390 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [req-bc6ce08c-7f0f-463e-884c-67c8ea03ba9f req-8cd75f4e-3844-4e67-a480-dc6e6952c536 service nova] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:32:10.774996 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [req-bc6ce08c-7f0f-463e-884c-67c8ea03ba9f req-8cd75f4e-3844-4e67-a480-dc6e6952c536 service nova] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:32:10.787974 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.compute.server_external_events [req-bc6ce08c-7f0f-463e-884c-67c8ea03ba9f req-8cd75f4e-3844-4e67-a480-dc6e6952c536 service nova] Creating event network-vif-deleted:d49ef841-e8bf-49f7-b85b-a7fbc34a1e23 for instance 2ad7bd6d-558b-4b12-a22f-67f33975e5fd on hv-cinder-84630 Aug 27 02:32:10.793320 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [req-bc6ce08c-7f0f-463e-884c-67c8ea03ba9f req-8cd75f4e-3844-4e67-a480-dc6e6952c536 service nova] 10.222.0.52 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.048650 Aug 27 02:32:10.798164 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 455/904] 10.222.0.52 () {64 vars in 1317 bytes} [Sat Aug 27 02:32:10 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 54 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Aug 27 02:32:11.265162 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-8ad03eb7-0e84-4d37-bccf-a00e0dcb4896 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:32:11.267161 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-8ad03eb7-0e84-4d37-bccf-a00e0dcb4896 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] [instance: 2ad7bd6d-558b-4b12-a22f-67f33975e5fd] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:32:11.274811 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-8ad03eb7-0e84-4d37-bccf-a00e0dcb4896 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:32:11.275464 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-8ad03eb7-0e84-4d37-bccf-a00e0dcb4896 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:32:11.275989 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-8ad03eb7-0e84-4d37-bccf-a00e0dcb4896 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:32:11.276789 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-8ad03eb7-0e84-4d37-bccf-a00e0dcb4896 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:32:11.278299 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-8ad03eb7-0e84-4d37-bccf-a00e0dcb4896 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:32:11.279223 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-8ad03eb7-0e84-4d37-bccf-a00e0dcb4896 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:32:11.314500 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-8ad03eb7-0e84-4d37-bccf-a00e0dcb4896 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:32:11.315592 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-8ad03eb7-0e84-4d37-bccf-a00e0dcb4896 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:32:11.316126 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-8ad03eb7-0e84-4d37-bccf-a00e0dcb4896 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:32:11.316799 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.objects.instance [None req-8ad03eb7-0e84-4d37-bccf-a00e0dcb4896 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lazy-loading 'fault' on Instance uuid 2ad7bd6d-558b-4b12-a22f-67f33975e5fd {{(pid=120863) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Aug 27 02:32:11.364854 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-8ad03eb7-0e84-4d37-bccf-a00e0dcb4896 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '26a86351583d49d090b3db09f127d49f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5d5ec8450c0941d18d63558baab54737', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:32:11.378302 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-8ad03eb7-0e84-4d37-bccf-a00e0dcb4896 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] 10.222.0.52 "GET /compute/v2.1/servers/2ad7bd6d-558b-4b12-a22f-67f33975e5fd" status: 200 len: 1330 microversion: 2.1 time: 0.118100 Aug 27 02:32:11.378904 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 450/905] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:32:11 2022] GET /compute/v2.1/servers/2ad7bd6d-558b-4b12-a22f-67f33975e5fd => generated 1330 bytes in 119 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:32:12.400672 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-0f811798-f1ea-4228-bc9c-62e87a3a2177 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Calling method '>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:32:12.402252 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-0f811798-f1ea-4228-bc9c-62e87a3a2177 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] [instance: 2ad7bd6d-558b-4b12-a22f-67f33975e5fd] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:32:12.414100 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-0f811798-f1ea-4228-bc9c-62e87a3a2177 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:32:12.414899 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-0f811798-f1ea-4228-bc9c-62e87a3a2177 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:32:12.415570 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-0f811798-f1ea-4228-bc9c-62e87a3a2177 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:32:12.416234 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-0f811798-f1ea-4228-bc9c-62e87a3a2177 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:32:12.416870 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-0f811798-f1ea-4228-bc9c-62e87a3a2177 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:32:12.417545 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-0f811798-f1ea-4228-bc9c-62e87a3a2177 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:32:12.437496 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.wsgi [None req-0f811798-f1ea-4228-bc9c-62e87a3a2177 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] HTTP exception thrown: Instance 2ad7bd6d-558b-4b12-a22f-67f33975e5fd could not be found. Aug 27 02:32:12.439630 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-0f811798-f1ea-4228-bc9c-62e87a3a2177 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Request method failure captured: Aug 27 02:32:12.439630 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: request: GET /compute/v2.1/servers/2ad7bd6d-558b-4b12-a22f-67f33975e5fd HTTP/1.1 Aug 27 02:32:12.439630 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: Accept: application/json Aug 27 02:32:12.439630 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: Accept-Encoding: identity Aug 27 02:32:12.439630 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: Connection: close Aug 27 02:32:12.439630 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: Content-Length: 0 Aug 27 02:32:12.439630 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: Content-Type: application/json Aug 27 02:32:12.439630 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: Host: 10.222.0.52 Aug 27 02:32:12.439630 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: Openstack-System-Scope: None Aug 27 02:32:12.439630 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: User-Agent: python-urllib3/1.26.11 Aug 27 02:32:12.439630 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-Auth-Token: gAAAAABjCX_pkOb4-Fffq6Hbe0cfjlj1ARL1JmORSzrYWucWmWgZ67LEorDD3IWpPkj3UP534US8GW_nja0iAKwcdxSx3e5teQzZMdrCvOvA5aSGJO8iodOjhfOIsl8L0d8om8kj8_3q-8uTHxKupSTnIQy_J-f1kC4e22VNejIIiExDsGitats Aug 27 02:32:12.439630 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-Domain-Id: None Aug 27 02:32:12.439630 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-Domain-Name: None Aug 27 02:32:12.439630 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-Identity-Status: Confirmed Aug 27 02:32:12.439630 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-Is-Admin-Project: True Aug 27 02:32:12.439630 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-Project-Domain-Id: default Aug 27 02:32:12.439630 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-Project-Domain-Name: Default Aug 27 02:32:12.439630 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-Project-Id: 5d5ec8450c0941d18d63558baab54737 Aug 27 02:32:12.439630 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-Project-Name: tempest-AttachVolumeNegativeTest-1988669364 Aug 27 02:32:12.439630 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-Role: reader,member Aug 27 02:32:12.439630 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-Roles: reader,member Aug 27 02:32:12.440932 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-Service-Catalog: [{"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.52:8080", "publicURL": "http://10.222.0.52:8080/v1/AUTH_5d5ec8450c0941d18d63558baab54737"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52:9696/networking"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/compute/v2/5d5ec8450c0941d18d63558baab54737"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/image"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/volume/v3/5d5ec8450c0941d18d63558baab54737"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/identity"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/volume/v3/5d5ec8450c0941d18d63558baab54737"}]}] Aug 27 02:32:12.440932 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-Tenant: tempest-AttachVolumeNegativeTest-1988669364 Aug 27 02:32:12.440932 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-Tenant-Id: 5d5ec8450c0941d18d63558baab54737 Aug 27 02:32:12.440932 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-1988669364 Aug 27 02:32:12.440932 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-User: tempest-AttachVolumeNegativeTest-1988669364-project Aug 27 02:32:12.440932 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-User-Domain-Id: default Aug 27 02:32:12.440932 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-User-Domain-Name: Default Aug 27 02:32:12.440932 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-User-Id: 26a86351583d49d090b3db09f127d49f Aug 27 02:32:12.440932 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-User-Name: tempest-AttachVolumeNegativeTest-1988669364-project Aug 27 02:32:12.440932 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: method: > Aug 27 02:32:12.440932 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: exception: Instance 2ad7bd6d-558b-4b12-a22f-67f33975e5fd could not be found. Aug 27 02:32:12.440932 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: action_args: {'id': '2ad7bd6d-558b-4b12-a22f-67f33975e5fd'} Aug 27 02:32:12.441692 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Aug 27 02:32:12.441692 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 27 02:32:12.441692 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Aug 27 02:32:12.441692 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Aug 27 02:32:12.441692 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Aug 27 02:32:12.441692 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Aug 27 02:32:12.441692 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Aug 27 02:32:12.441692 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Aug 27 02:32:12.441692 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Aug 27 02:32:12.441692 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Aug 27 02:32:12.441692 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 2ad7bd6d-558b-4b12-a22f-67f33975e5fd could not be found. Aug 27 02:32:12.441692 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi Aug 27 02:32:12.441692 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 27 02:32:12.441692 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi Aug 27 02:32:12.441692 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 27 02:32:12.441692 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 27 02:32:12.441692 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 27 02:32:12.441692 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Aug 27 02:32:12.441692 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Aug 27 02:32:12.441692 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Aug 27 02:32:12.442947 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Aug 27 02:32:12.442947 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Aug 27 02:32:12.442947 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi instance = self._get_server( Aug 27 02:32:12.442947 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Aug 27 02:32:12.442947 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Aug 27 02:32:12.442947 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Aug 27 02:32:12.442947 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Aug 27 02:32:12.442947 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 2ad7bd6d-558b-4b12-a22f-67f33975e5fd could not be found. Aug 27 02:32:12.442947 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi Aug 27 02:32:12.442947 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 27 02:32:12.442947 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi Aug 27 02:32:12.442947 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 27 02:32:12.442947 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 27 02:32:12.442947 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 27 02:32:12.442947 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Aug 27 02:32:12.442947 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Aug 27 02:32:12.442947 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 2ad7bd6d-558b-4b12-a22f-67f33975e5fd could not be found. Aug 27 02:32:12.442947 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi Aug 27 02:32:12.448845 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-0f811798-f1ea-4228-bc9c-62e87a3a2177 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Returning 404 to user: Instance 2ad7bd6d-558b-4b12-a22f-67f33975e5fd could not be found. {{(pid=120864) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Aug 27 02:32:12.450231 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-0f811798-f1ea-4228-bc9c-62e87a3a2177 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] 10.222.0.52 "GET /compute/v2.1/servers/2ad7bd6d-558b-4b12-a22f-67f33975e5fd" status: 404 len: 111 microversion: 2.1 time: 0.053808 Aug 27 02:32:12.451009 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 456/906] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:32:12 2022] GET /compute/v2.1/servers/2ad7bd6d-558b-4b12-a22f-67f33975e5fd => generated 111 bytes in 55 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Aug 27 02:32:12.461101 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-65e91e94-6541-419d-b3a5-1afc4a61dd38 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Calling method '.version_select at 0x7f6b1b0e2040>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:32:12.491364 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-65e91e94-6541-419d-b3a5-1afc4a61dd38 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] 10.222.0.52 "DELETE /compute/v2.1/os-keypairs/tempest-keypair-527069421" status: 202 len: 0 microversion: 2.1 time: 0.033165 Aug 27 02:32:12.492085 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 451/907] 10.222.0.52 () {58 vars in 1218 bytes} [Sat Aug 27 02:32:12 2022] DELETE /compute/v2.1/os-keypairs/tempest-keypair-527069421 => generated 0 bytes in 36 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Aug 27 02:32:13.455755 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-5e9da2fb-e29f-4c7b-9b3e-14e4631bd1d2 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Action: 'create', calling method: .version_select at 0x7f6b1e8588b0>, body: {"keypair": {"name": "tempest-keypair-835171988", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBJ7V7+a9OCM2nSVIj/+sFmWpB5IqI5hQ84VB1vVdX1152RMLxKnkgzYR2yB2M7Ra1LP1aTtuojNJoVCQcbzjmofb/HGs/m8qnJDP0sAeWC12A8w5AluvtjM9QzYTHzOwhg=="}} {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 27 02:32:13.491908 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.quota [None req-5e9da2fb-e29f-4c7b-9b3e-14e4631bd1d2 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Getting quotas for project 5d5ec8450c0941d18d63558baab54737. Resources: {'key_pairs'} {{(pid=120864) _get_quotas /opt/stack/nova/nova/quota.py:393}} Aug 27 02:32:13.502562 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.quota [None req-5e9da2fb-e29f-4c7b-9b3e-14e4631bd1d2 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Getting quotas for user 26a86351583d49d090b3db09f127d49f and project 5d5ec8450c0941d18d63558baab54737. Resources: {'key_pairs'} {{(pid=120864) _get_quotas /opt/stack/nova/nova/quota.py:383}} Aug 27 02:32:13.539644 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-5e9da2fb-e29f-4c7b-9b3e-14e4631bd1d2 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] 10.222.0.52 "POST /compute/v2.1/os-keypairs" status: 200 len: 385 microversion: 2.1 time: 0.087762 Aug 27 02:32:13.540671 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 457/908] 10.222.0.52 () {60 vars in 1159 bytes} [Sat Aug 27 02:32:13 2022] POST /compute/v2.1/os-keypairs => generated 385 bytes in 89 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 27 02:32:15.144903 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-532e5ca7-6c28-4523-8a6a-b72631fb04bc tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-AttachVolumeNegativeTest-server-1970593323", "imageRef": "9fc30b36-b827-46f9-892d-30e808bc2b8a", "flavorRef": "452", "adminPass": "***", "networks": [{"uuid": "134184f4-d632-4173-b77c-1566bf4eb0fb"}], "security_groups": [{"name": "tempest-securitygroup--598984941"}], "key_name": "tempest-keypair-835171988", "user_data": "}} {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 27 02:32:15.266208 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.network.neutron [None req-532e5ca7-6c28-4523-8a6a-b72631fb04bc tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] validate_networks() for [('134184f4-d632-4173-b77c-1566bf4eb0fb', None, None, None, None, None)] {{(pid=120863) validate_networks /opt/stack/nova/nova/network/neutron.py:2626}} Aug 27 02:32:15.682147 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.virt.hardware [None req-532e5ca7-6c28-4523-8a6a-b72631fb04bc tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Flavor limits 0:0:0 {{(pid=120863) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Aug 27 02:32:15.682580 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.virt.hardware [None req-532e5ca7-6c28-4523-8a6a-b72631fb04bc tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Image limits 0:0:0 {{(pid=120863) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Aug 27 02:32:15.683061 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.virt.hardware [None req-532e5ca7-6c28-4523-8a6a-b72631fb04bc tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Flavor pref 0:0:0 {{(pid=120863) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Aug 27 02:32:15.683538 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.virt.hardware [None req-532e5ca7-6c28-4523-8a6a-b72631fb04bc tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Image pref 0:0:0 {{(pid=120863) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Aug 27 02:32:15.684049 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.virt.hardware [None req-532e5ca7-6c28-4523-8a6a-b72631fb04bc tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=120863) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Aug 27 02:32:15.693854 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-532e5ca7-6c28-4523-8a6a-b72631fb04bc tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:32:15.694332 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-532e5ca7-6c28-4523-8a6a-b72631fb04bc tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:32:15.694770 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-532e5ca7-6c28-4523-8a6a-b72631fb04bc tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:32:15.695503 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-532e5ca7-6c28-4523-8a6a-b72631fb04bc tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:32:15.695960 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-532e5ca7-6c28-4523-8a6a-b72631fb04bc tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:32:15.696387 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-532e5ca7-6c28-4523-8a6a-b72631fb04bc tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:32:15.714569 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.quota [None req-532e5ca7-6c28-4523-8a6a-b72631fb04bc tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Getting quotas for project 5d5ec8450c0941d18d63558baab54737. Resources: {'cores', 'ram', 'instances'} {{(pid=120863) _get_quotas /opt/stack/nova/nova/quota.py:393}} Aug 27 02:32:15.719638 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.quota [None req-532e5ca7-6c28-4523-8a6a-b72631fb04bc tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Getting quotas for user 26a86351583d49d090b3db09f127d49f and project 5d5ec8450c0941d18d63558baab54737. Resources: {'cores', 'ram', 'instances'} {{(pid=120863) _get_quotas /opt/stack/nova/nova/quota.py:383}} Aug 27 02:32:15.732159 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-532e5ca7-6c28-4523-8a6a-b72631fb04bc tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Going to run 1 instances... {{(pid=120863) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Aug 27 02:32:15.734498 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-532e5ca7-6c28-4523-8a6a-b72631fb04bc tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] [instance: 77068029-afdb-4560-8d56-76c56b85ef3a] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='9fc30b36-b827-46f9-892d-30e808bc2b8a',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=120863) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Aug 27 02:32:15.790569 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-532e5ca7-6c28-4523-8a6a-b72631fb04bc tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] 10.222.0.52 "POST /compute/v2.1/servers" status: 202 len: 401 microversion: 2.1 time: 0.650448 Aug 27 02:32:15.791222 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 452/909] 10.222.0.52 () {60 vars in 1147 bytes} [Sat Aug 27 02:32:15 2022] POST /compute/v2.1/servers => generated 401 bytes in 652 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Aug 27 02:32:15.803362 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-d78dbf8f-ce25-414b-bae6-67ea19bc357e tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Calling method '>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:32:15.803961 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-d78dbf8f-ce25-414b-bae6-67ea19bc357e tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] [instance: 77068029-afdb-4560-8d56-76c56b85ef3a] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:32:15.861620 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-d78dbf8f-ce25-414b-bae6-67ea19bc357e tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '26a86351583d49d090b3db09f127d49f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5d5ec8450c0941d18d63558baab54737', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:32:15.869825 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-d78dbf8f-ce25-414b-bae6-67ea19bc357e tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] 10.222.0.52 "GET /compute/v2.1/servers/77068029-afdb-4560-8d56-76c56b85ef3a" status: 200 len: 1241 microversion: 2.1 time: 0.071105 Aug 27 02:32:15.870680 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 458/910] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:32:15 2022] GET /compute/v2.1/servers/77068029-afdb-4560-8d56-76c56b85ef3a => generated 1241 bytes in 73 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:32:16.892652 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-196015c0-b630-4230-baf7-e408d67bbefe tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:32:16.893512 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-196015c0-b630-4230-baf7-e408d67bbefe tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] [instance: 77068029-afdb-4560-8d56-76c56b85ef3a] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:32:16.900468 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-196015c0-b630-4230-baf7-e408d67bbefe tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:32:16.900939 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-196015c0-b630-4230-baf7-e408d67bbefe tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:32:16.901645 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-196015c0-b630-4230-baf7-e408d67bbefe tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:32:16.902357 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-196015c0-b630-4230-baf7-e408d67bbefe tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:32:16.902866 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-196015c0-b630-4230-baf7-e408d67bbefe tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:32:16.903661 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-196015c0-b630-4230-baf7-e408d67bbefe tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:32:16.974734 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-196015c0-b630-4230-baf7-e408d67bbefe tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '26a86351583d49d090b3db09f127d49f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5d5ec8450c0941d18d63558baab54737', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:32:16.984012 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-196015c0-b630-4230-baf7-e408d67bbefe tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] 10.222.0.52 "GET /compute/v2.1/servers/77068029-afdb-4560-8d56-76c56b85ef3a" status: 200 len: 1293 microversion: 2.1 time: 0.096804 Aug 27 02:32:16.985210 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 453/911] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:32:16 2022] GET /compute/v2.1/servers/77068029-afdb-4560-8d56-76c56b85ef3a => generated 1293 bytes in 99 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:32:18.005481 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-9da03645-a4a9-44d2-a1df-a448e7629cfd tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Calling method '>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:32:18.006045 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-9da03645-a4a9-44d2-a1df-a448e7629cfd tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] [instance: 77068029-afdb-4560-8d56-76c56b85ef3a] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:32:18.012846 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-9da03645-a4a9-44d2-a1df-a448e7629cfd tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:32:18.013481 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-9da03645-a4a9-44d2-a1df-a448e7629cfd tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:32:18.013929 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-9da03645-a4a9-44d2-a1df-a448e7629cfd tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:32:18.014616 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-9da03645-a4a9-44d2-a1df-a448e7629cfd tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:32:18.015028 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-9da03645-a4a9-44d2-a1df-a448e7629cfd tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:32:18.015657 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-9da03645-a4a9-44d2-a1df-a448e7629cfd tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:32:18.511956 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-9da03645-a4a9-44d2-a1df-a448e7629cfd tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '26a86351583d49d090b3db09f127d49f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5d5ec8450c0941d18d63558baab54737', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:32:18.521132 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-9da03645-a4a9-44d2-a1df-a448e7629cfd tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] 10.222.0.52 "GET /compute/v2.1/servers/77068029-afdb-4560-8d56-76c56b85ef3a" status: 200 len: 1366 microversion: 2.1 time: 0.519494 Aug 27 02:32:18.521721 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 459/912] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:32:18 2022] GET /compute/v2.1/servers/77068029-afdb-4560-8d56-76c56b85ef3a => generated 1366 bytes in 521 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:32:18.843416 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [req-e52639ab-e0c0-4571-9f4e-d53a6893e42f req-10bd54ac-6878-49f6-bec5-accc70c8c2f9 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "77068029-afdb-4560-8d56-76c56b85ef3a", "tag": "46a7cd9a-679a-4cbc-95b2-41424b3783c2"}]} {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 27 02:32:18.855750 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [req-e52639ab-e0c0-4571-9f4e-d53a6893e42f req-10bd54ac-6878-49f6-bec5-accc70c8c2f9 service nova] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:32:18.856966 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [req-e52639ab-e0c0-4571-9f4e-d53a6893e42f req-10bd54ac-6878-49f6-bec5-accc70c8c2f9 service nova] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:32:18.857503 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [req-e52639ab-e0c0-4571-9f4e-d53a6893e42f req-10bd54ac-6878-49f6-bec5-accc70c8c2f9 service nova] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:32:18.868932 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.compute.server_external_events [req-e52639ab-e0c0-4571-9f4e-d53a6893e42f req-10bd54ac-6878-49f6-bec5-accc70c8c2f9 service nova] Creating event network-changed:46a7cd9a-679a-4cbc-95b2-41424b3783c2 for instance 77068029-afdb-4560-8d56-76c56b85ef3a on hv-cinder-84630 Aug 27 02:32:18.874840 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [req-e52639ab-e0c0-4571-9f4e-d53a6893e42f req-10bd54ac-6878-49f6-bec5-accc70c8c2f9 service nova] 10.222.0.52 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.037079 Aug 27 02:32:18.875643 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 454/913] 10.222.0.52 () {64 vars in 1317 bytes} [Sat Aug 27 02:32:18 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 38 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 27 02:32:19.553745 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-283c0dc0-e89c-4650-bfa8-50b721a16c3f tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Calling method '>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:32:19.554312 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-283c0dc0-e89c-4650-bfa8-50b721a16c3f tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] [instance: 77068029-afdb-4560-8d56-76c56b85ef3a] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:32:19.562565 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-283c0dc0-e89c-4650-bfa8-50b721a16c3f tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:32:19.563057 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-283c0dc0-e89c-4650-bfa8-50b721a16c3f tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:32:19.563490 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-283c0dc0-e89c-4650-bfa8-50b721a16c3f tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:32:19.564195 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-283c0dc0-e89c-4650-bfa8-50b721a16c3f tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:32:19.564626 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-283c0dc0-e89c-4650-bfa8-50b721a16c3f tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:32:19.565066 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-283c0dc0-e89c-4650-bfa8-50b721a16c3f tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:32:19.714869 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-283c0dc0-e89c-4650-bfa8-50b721a16c3f tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '26a86351583d49d090b3db09f127d49f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5d5ec8450c0941d18d63558baab54737', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:32:19.727117 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-283c0dc0-e89c-4650-bfa8-50b721a16c3f tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] 10.222.0.52 "GET /compute/v2.1/servers/77068029-afdb-4560-8d56-76c56b85ef3a" status: 200 len: 1366 microversion: 2.1 time: 0.180324 Aug 27 02:32:19.727938 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 460/914] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:32:19 2022] GET /compute/v2.1/servers/77068029-afdb-4560-8d56-76c56b85ef3a => generated 1366 bytes in 182 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:32:20.753954 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-6fb0f694-68d2-410a-8c62-57cbcec421bd tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:32:20.754732 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-6fb0f694-68d2-410a-8c62-57cbcec421bd tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] [instance: 77068029-afdb-4560-8d56-76c56b85ef3a] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:32:20.763878 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-6fb0f694-68d2-410a-8c62-57cbcec421bd tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:32:20.764362 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-6fb0f694-68d2-410a-8c62-57cbcec421bd tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:32:20.764797 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-6fb0f694-68d2-410a-8c62-57cbcec421bd tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:32:20.765572 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-6fb0f694-68d2-410a-8c62-57cbcec421bd tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:32:20.765973 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-6fb0f694-68d2-410a-8c62-57cbcec421bd tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:32:20.766395 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-6fb0f694-68d2-410a-8c62-57cbcec421bd tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:32:20.924268 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-6fb0f694-68d2-410a-8c62-57cbcec421bd tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '26a86351583d49d090b3db09f127d49f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5d5ec8450c0941d18d63558baab54737', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:32:20.934348 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-6fb0f694-68d2-410a-8c62-57cbcec421bd tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] 10.222.0.52 "GET /compute/v2.1/servers/77068029-afdb-4560-8d56-76c56b85ef3a" status: 200 len: 1366 microversion: 2.1 time: 0.187696 Aug 27 02:32:20.937274 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 455/915] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:32:20 2022] GET /compute/v2.1/servers/77068029-afdb-4560-8d56-76c56b85ef3a => generated 1366 bytes in 192 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:32:21.956224 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-8257dea7-f600-491f-b183-cf09205cac9f tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Calling method '>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:32:21.956840 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-8257dea7-f600-491f-b183-cf09205cac9f tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] [instance: 77068029-afdb-4560-8d56-76c56b85ef3a] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:32:21.963913 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-8257dea7-f600-491f-b183-cf09205cac9f tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:32:21.964391 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-8257dea7-f600-491f-b183-cf09205cac9f tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:32:21.964819 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-8257dea7-f600-491f-b183-cf09205cac9f tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:32:21.965633 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-8257dea7-f600-491f-b183-cf09205cac9f tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:32:21.966249 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-8257dea7-f600-491f-b183-cf09205cac9f tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:32:21.966782 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-8257dea7-f600-491f-b183-cf09205cac9f tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:32:22.116050 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-8257dea7-f600-491f-b183-cf09205cac9f tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '26a86351583d49d090b3db09f127d49f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5d5ec8450c0941d18d63558baab54737', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:32:22.127893 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-8257dea7-f600-491f-b183-cf09205cac9f tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] 10.222.0.52 "GET /compute/v2.1/servers/77068029-afdb-4560-8d56-76c56b85ef3a" status: 200 len: 1366 microversion: 2.1 time: 0.175134 Aug 27 02:32:22.128569 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 461/916] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:32:21 2022] GET /compute/v2.1/servers/77068029-afdb-4560-8d56-76c56b85ef3a => generated 1366 bytes in 177 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:32:23.152289 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-6daaac6a-c213-4f47-9121-eed6ee302d57 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:32:23.153113 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-6daaac6a-c213-4f47-9121-eed6ee302d57 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] [instance: 77068029-afdb-4560-8d56-76c56b85ef3a] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:32:23.160095 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-6daaac6a-c213-4f47-9121-eed6ee302d57 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:32:23.160586 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-6daaac6a-c213-4f47-9121-eed6ee302d57 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:32:23.161024 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-6daaac6a-c213-4f47-9121-eed6ee302d57 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:32:23.161797 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-6daaac6a-c213-4f47-9121-eed6ee302d57 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:32:23.162247 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-6daaac6a-c213-4f47-9121-eed6ee302d57 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:32:23.162742 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-6daaac6a-c213-4f47-9121-eed6ee302d57 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:32:23.304738 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-6daaac6a-c213-4f47-9121-eed6ee302d57 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '26a86351583d49d090b3db09f127d49f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5d5ec8450c0941d18d63558baab54737', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:32:23.313262 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-6daaac6a-c213-4f47-9121-eed6ee302d57 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] 10.222.0.52 "GET /compute/v2.1/servers/77068029-afdb-4560-8d56-76c56b85ef3a" status: 200 len: 1366 microversion: 2.1 time: 0.164760 Aug 27 02:32:23.313856 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 456/917] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:32:23 2022] GET /compute/v2.1/servers/77068029-afdb-4560-8d56-76c56b85ef3a => generated 1366 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:32:24.340259 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-12b38ab9-e41c-4bcd-aee9-241eac8a7dbf tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Calling method '>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:32:24.341581 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-12b38ab9-e41c-4bcd-aee9-241eac8a7dbf tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] [instance: 77068029-afdb-4560-8d56-76c56b85ef3a] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:32:24.344341 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG dbcounter [-] [120864] Writing DB stats nova_api:SELECT=56,nova_api:DELETE=1,nova_api:UPDATE=1,nova_api:INSERT=1 {{(pid=120864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 27 02:32:24.350244 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-12b38ab9-e41c-4bcd-aee9-241eac8a7dbf tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:32:24.350729 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-12b38ab9-e41c-4bcd-aee9-241eac8a7dbf tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:32:24.351162 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-12b38ab9-e41c-4bcd-aee9-241eac8a7dbf tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:32:24.351888 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-12b38ab9-e41c-4bcd-aee9-241eac8a7dbf tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:32:24.352340 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-12b38ab9-e41c-4bcd-aee9-241eac8a7dbf tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:32:24.352756 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-12b38ab9-e41c-4bcd-aee9-241eac8a7dbf tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:32:24.356715 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG dbcounter [-] [120864] Writing DB stats nova_cell1:SELECT=88,nova_cell1:UPDATE=3,nova_cell1:INSERT=2 {{(pid=120864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 27 02:32:24.597234 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-12b38ab9-e41c-4bcd-aee9-241eac8a7dbf tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '26a86351583d49d090b3db09f127d49f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5d5ec8450c0941d18d63558baab54737', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:32:24.605243 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-12b38ab9-e41c-4bcd-aee9-241eac8a7dbf tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] 10.222.0.52 "GET /compute/v2.1/servers/77068029-afdb-4560-8d56-76c56b85ef3a" status: 200 len: 1366 microversion: 2.1 time: 0.268782 Aug 27 02:32:24.605824 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 462/918] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:32:24 2022] GET /compute/v2.1/servers/77068029-afdb-4560-8d56-76c56b85ef3a => generated 1366 bytes in 270 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:32:25.630356 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-387dfd5a-ad20-4d03-bf8c-8d793503ca35 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:32:25.630356 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-387dfd5a-ad20-4d03-bf8c-8d793503ca35 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] [instance: 77068029-afdb-4560-8d56-76c56b85ef3a] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:32:25.631076 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG dbcounter [-] [120863] Writing DB stats nova_api:SELECT=54,nova_api:UPDATE=1,nova_api:DELETE=1,nova_api:INSERT=3 {{(pid=120863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 27 02:32:25.636168 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-387dfd5a-ad20-4d03-bf8c-8d793503ca35 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:32:25.636666 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-387dfd5a-ad20-4d03-bf8c-8d793503ca35 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:32:25.637105 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-387dfd5a-ad20-4d03-bf8c-8d793503ca35 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:32:25.637838 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-387dfd5a-ad20-4d03-bf8c-8d793503ca35 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:32:25.638292 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-387dfd5a-ad20-4d03-bf8c-8d793503ca35 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:32:25.638734 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-387dfd5a-ad20-4d03-bf8c-8d793503ca35 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:32:25.642124 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG dbcounter [-] [120863] Writing DB stats nova_cell1:SELECT=92,nova_cell1:UPDATE=1,nova_cell1:INSERT=2 {{(pid=120863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 27 02:32:25.702660 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG dbcounter [-] [120863] Writing DB stats nova_cell0:SELECT=2 {{(pid=120863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 27 02:32:25.728477 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG dbcounter [-] [120863] Writing DB stats nova_cell0:SELECT=6 {{(pid=120863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 27 02:32:25.777052 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-387dfd5a-ad20-4d03-bf8c-8d793503ca35 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '26a86351583d49d090b3db09f127d49f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5d5ec8450c0941d18d63558baab54737', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:32:25.785420 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-387dfd5a-ad20-4d03-bf8c-8d793503ca35 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] 10.222.0.52 "GET /compute/v2.1/servers/77068029-afdb-4560-8d56-76c56b85ef3a" status: 200 len: 1366 microversion: 2.1 time: 0.161776 Aug 27 02:32:25.786025 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 457/919] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:32:25 2022] GET /compute/v2.1/servers/77068029-afdb-4560-8d56-76c56b85ef3a => generated 1366 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 27 02:32:26.807133 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-fd8c2871-b0c2-4248-bb46-be24bef2bd51 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Calling method '>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:32:26.807769 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-fd8c2871-b0c2-4248-bb46-be24bef2bd51 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] [instance: 77068029-afdb-4560-8d56-76c56b85ef3a] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:32:26.810146 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG dbcounter [-] [120864] Writing DB stats nova_cell0:SELECT=6 {{(pid=120864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 27 02:32:26.815805 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-fd8c2871-b0c2-4248-bb46-be24bef2bd51 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:32:26.816306 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-fd8c2871-b0c2-4248-bb46-be24bef2bd51 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:32:26.816739 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-fd8c2871-b0c2-4248-bb46-be24bef2bd51 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:32:26.817458 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-fd8c2871-b0c2-4248-bb46-be24bef2bd51 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:32:26.817905 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-fd8c2871-b0c2-4248-bb46-be24bef2bd51 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:32:26.818330 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-fd8c2871-b0c2-4248-bb46-be24bef2bd51 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:32:26.959561 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-fd8c2871-b0c2-4248-bb46-be24bef2bd51 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '26a86351583d49d090b3db09f127d49f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5d5ec8450c0941d18d63558baab54737', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:32:26.967618 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-fd8c2871-b0c2-4248-bb46-be24bef2bd51 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] 10.222.0.52 "GET /compute/v2.1/servers/77068029-afdb-4560-8d56-76c56b85ef3a" status: 200 len: 1366 microversion: 2.1 time: 0.164399 Aug 27 02:32:26.968514 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 463/920] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:32:26 2022] GET /compute/v2.1/servers/77068029-afdb-4560-8d56-76c56b85ef3a => generated 1366 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:32:27.010121 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [req-4bd29072-3344-453d-8cd7-55622c70b557 req-5956eae9-ee99-48bd-bbba-b9dd2c1bd2b2 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "77068029-afdb-4560-8d56-76c56b85ef3a", "name": "network-vif-plugged", "status": "completed", "tag": "46a7cd9a-679a-4cbc-95b2-41424b3783c2"}]} {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 27 02:32:27.020474 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [req-4bd29072-3344-453d-8cd7-55622c70b557 req-5956eae9-ee99-48bd-bbba-b9dd2c1bd2b2 service nova] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:32:27.020959 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [req-4bd29072-3344-453d-8cd7-55622c70b557 req-5956eae9-ee99-48bd-bbba-b9dd2c1bd2b2 service nova] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:32:27.021447 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [req-4bd29072-3344-453d-8cd7-55622c70b557 req-5956eae9-ee99-48bd-bbba-b9dd2c1bd2b2 service nova] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:32:27.032975 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.compute.server_external_events [req-4bd29072-3344-453d-8cd7-55622c70b557 req-5956eae9-ee99-48bd-bbba-b9dd2c1bd2b2 service nova] Creating event network-vif-plugged:46a7cd9a-679a-4cbc-95b2-41424b3783c2 for instance 77068029-afdb-4560-8d56-76c56b85ef3a on hv-cinder-84630 Aug 27 02:32:27.037842 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [req-4bd29072-3344-453d-8cd7-55622c70b557 req-5956eae9-ee99-48bd-bbba-b9dd2c1bd2b2 service nova] 10.222.0.52 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.031421 Aug 27 02:32:27.038450 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 458/921] 10.222.0.52 () {64 vars in 1317 bytes} [Sat Aug 27 02:32:27 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 33 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 27 02:32:27.999525 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-53d81c4c-7055-429c-8706-65912d62163a tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Calling method '>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:32:28.000167 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-53d81c4c-7055-429c-8706-65912d62163a tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] [instance: 77068029-afdb-4560-8d56-76c56b85ef3a] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:32:28.007479 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-53d81c4c-7055-429c-8706-65912d62163a tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:32:28.008154 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-53d81c4c-7055-429c-8706-65912d62163a tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:32:28.008636 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-53d81c4c-7055-429c-8706-65912d62163a tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:32:28.011334 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-53d81c4c-7055-429c-8706-65912d62163a tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:32:28.011880 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-53d81c4c-7055-429c-8706-65912d62163a tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:32:28.012354 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-53d81c4c-7055-429c-8706-65912d62163a tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:32:28.171523 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-53d81c4c-7055-429c-8706-65912d62163a tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '26a86351583d49d090b3db09f127d49f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5d5ec8450c0941d18d63558baab54737', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:32:28.179840 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-53d81c4c-7055-429c-8706-65912d62163a tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] 10.222.0.52 "GET /compute/v2.1/servers/77068029-afdb-4560-8d56-76c56b85ef3a" status: 200 len: 1366 microversion: 2.1 time: 0.185617 Aug 27 02:32:28.180427 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 464/922] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:32:27 2022] GET /compute/v2.1/servers/77068029-afdb-4560-8d56-76c56b85ef3a => generated 1366 bytes in 188 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:32:29.202922 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-93b9c901-cccf-47b2-8ca7-72ae2b424d9f tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:32:29.203761 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-93b9c901-cccf-47b2-8ca7-72ae2b424d9f tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] [instance: 77068029-afdb-4560-8d56-76c56b85ef3a] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:32:29.210577 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-93b9c901-cccf-47b2-8ca7-72ae2b424d9f tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:32:29.211181 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-93b9c901-cccf-47b2-8ca7-72ae2b424d9f tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:32:29.211649 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-93b9c901-cccf-47b2-8ca7-72ae2b424d9f tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:32:29.212627 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-93b9c901-cccf-47b2-8ca7-72ae2b424d9f tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:32:29.213152 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-93b9c901-cccf-47b2-8ca7-72ae2b424d9f tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:32:29.213662 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-93b9c901-cccf-47b2-8ca7-72ae2b424d9f tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:32:29.361603 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-93b9c901-cccf-47b2-8ca7-72ae2b424d9f tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '26a86351583d49d090b3db09f127d49f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5d5ec8450c0941d18d63558baab54737', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:32:29.369695 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-93b9c901-cccf-47b2-8ca7-72ae2b424d9f tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] 10.222.0.52 "GET /compute/v2.1/servers/77068029-afdb-4560-8d56-76c56b85ef3a" status: 200 len: 1366 microversion: 2.1 time: 0.171838 Aug 27 02:32:29.370270 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 459/923] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:32:29 2022] GET /compute/v2.1/servers/77068029-afdb-4560-8d56-76c56b85ef3a => generated 1366 bytes in 173 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:32:30.391352 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-213c7020-1f98-4623-b848-2d1296d87034 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Calling method '>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:32:30.391959 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-213c7020-1f98-4623-b848-2d1296d87034 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] [instance: 77068029-afdb-4560-8d56-76c56b85ef3a] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:32:30.405170 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-213c7020-1f98-4623-b848-2d1296d87034 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:32:30.405488 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-213c7020-1f98-4623-b848-2d1296d87034 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:32:30.405899 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-213c7020-1f98-4623-b848-2d1296d87034 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:32:30.406600 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-213c7020-1f98-4623-b848-2d1296d87034 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:32:30.407119 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-213c7020-1f98-4623-b848-2d1296d87034 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:32:30.407418 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-213c7020-1f98-4623-b848-2d1296d87034 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:32:30.543449 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-213c7020-1f98-4623-b848-2d1296d87034 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '26a86351583d49d090b3db09f127d49f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5d5ec8450c0941d18d63558baab54737', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:32:30.551828 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-213c7020-1f98-4623-b848-2d1296d87034 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] 10.222.0.52 "GET /compute/v2.1/servers/77068029-afdb-4560-8d56-76c56b85ef3a" status: 200 len: 1556 microversion: 2.1 time: 0.164793 Aug 27 02:32:30.552861 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 465/924] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:32:30 2022] GET /compute/v2.1/servers/77068029-afdb-4560-8d56-76c56b85ef3a => generated 1556 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:32:30.572836 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-b5c5bbc9-6e6f-4c7a-a307-1840fb664ffe tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:32:30.573446 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-b5c5bbc9-6e6f-4c7a-a307-1840fb664ffe tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] [instance: 77068029-afdb-4560-8d56-76c56b85ef3a] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:32:30.579650 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-b5c5bbc9-6e6f-4c7a-a307-1840fb664ffe tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:32:30.580140 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-b5c5bbc9-6e6f-4c7a-a307-1840fb664ffe tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:32:30.580574 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-b5c5bbc9-6e6f-4c7a-a307-1840fb664ffe tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:32:30.581301 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-b5c5bbc9-6e6f-4c7a-a307-1840fb664ffe tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:32:30.581755 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-b5c5bbc9-6e6f-4c7a-a307-1840fb664ffe tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:32:30.582185 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-b5c5bbc9-6e6f-4c7a-a307-1840fb664ffe tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:32:30.662654 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-b5c5bbc9-6e6f-4c7a-a307-1840fb664ffe tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] 10.222.0.52 "GET /compute/v2.1/servers/77068029-afdb-4560-8d56-76c56b85ef3a/os-interface" status: 200 len: 285 microversion: 2.1 time: 0.093859 Aug 27 02:32:30.663412 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 460/925] 10.222.0.52 () {58 vars in 1275 bytes} [Sat Aug 27 02:32:30 2022] GET /compute/v2.1/servers/77068029-afdb-4560-8d56-76c56b85ef3a/os-interface => generated 285 bytes in 95 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 27 02:32:31.698784 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [req-75036edf-55be-4005-a35b-12aee69d6b12 req-e445497d-4fc8-4496-a6b5-0a7648e836b1 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "77068029-afdb-4560-8d56-76c56b85ef3a", "tag": "46a7cd9a-679a-4cbc-95b2-41424b3783c2"}]} {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 27 02:32:31.710234 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [req-75036edf-55be-4005-a35b-12aee69d6b12 req-e445497d-4fc8-4496-a6b5-0a7648e836b1 service nova] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:32:31.710974 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [req-75036edf-55be-4005-a35b-12aee69d6b12 req-e445497d-4fc8-4496-a6b5-0a7648e836b1 service nova] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:32:31.711586 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [req-75036edf-55be-4005-a35b-12aee69d6b12 req-e445497d-4fc8-4496-a6b5-0a7648e836b1 service nova] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:32:31.727248 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.compute.server_external_events [req-75036edf-55be-4005-a35b-12aee69d6b12 req-e445497d-4fc8-4496-a6b5-0a7648e836b1 service nova] Creating event network-changed:46a7cd9a-679a-4cbc-95b2-41424b3783c2 for instance 77068029-afdb-4560-8d56-76c56b85ef3a on hv-cinder-84630 Aug 27 02:32:31.733802 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [req-75036edf-55be-4005-a35b-12aee69d6b12 req-e445497d-4fc8-4496-a6b5-0a7648e836b1 service nova] 10.222.0.52 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.038462 Aug 27 02:32:31.734781 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 466/926] 10.222.0.52 () {64 vars in 1317 bytes} [Sat Aug 27 02:32:31 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 40 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 27 02:32:51.257723 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG dbcounter [-] [120863] Writing DB stats nova_api:SELECT=7 {{(pid=120863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 27 02:32:51.258540 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG dbcounter [-] [120863] Writing DB stats nova_cell1:SELECT=17 {{(pid=120863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 27 02:32:51.263388 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-f05fced4-6eea-4cae-9ae4-50869ca56fa0 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:32:51.264298 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-f05fced4-6eea-4cae-9ae4-50869ca56fa0 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] [instance: fff5c233-ad7f-4022-8582-e8620f32ebd5] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:32:51.271591 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-f05fced4-6eea-4cae-9ae4-50869ca56fa0 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:32:51.272146 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-f05fced4-6eea-4cae-9ae4-50869ca56fa0 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:32:51.272903 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-f05fced4-6eea-4cae-9ae4-50869ca56fa0 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:32:51.273862 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-f05fced4-6eea-4cae-9ae4-50869ca56fa0 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:32:51.274507 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-f05fced4-6eea-4cae-9ae4-50869ca56fa0 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:32:51.275133 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-f05fced4-6eea-4cae-9ae4-50869ca56fa0 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:32:51.302819 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-f05fced4-6eea-4cae-9ae4-50869ca56fa0 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] 10.222.0.52 "GET /compute/v2.1/servers/fff5c233-ad7f-4022-8582-e8620f32ebd5/ips" status: 200 len: 152 microversion: 2.20 time: 0.046532 Aug 27 02:32:51.303412 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 461/927] 10.222.0.52 () {60 vars in 1289 bytes} [Sat Aug 27 02:32:51 2022] GET /compute/v2.1/servers/fff5c233-ad7f-4022-8582-e8620f32ebd5/ips => generated 152 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Aug 27 02:32:53.230060 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG dbcounter [-] [120864] Writing DB stats nova_api:SELECT=9 {{(pid=120864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 27 02:32:53.230810 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG dbcounter [-] [120864] Writing DB stats nova_cell1:SELECT=25 {{(pid=120864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 27 02:32:53.235770 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-0c8de93b-08e0-419c-a285-afbbfb688c26 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Action: 'action', calling method: >, body: {"shelve": {}} {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 27 02:32:53.236461 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-0c8de93b-08e0-419c-a285-afbbfb688c26 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] [instance: fff5c233-ad7f-4022-8582-e8620f32ebd5] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:32:53.244112 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-0c8de93b-08e0-419c-a285-afbbfb688c26 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:32:53.244588 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-0c8de93b-08e0-419c-a285-afbbfb688c26 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:32:53.245048 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-0c8de93b-08e0-419c-a285-afbbfb688c26 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:32:53.245853 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-0c8de93b-08e0-419c-a285-afbbfb688c26 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:32:53.246305 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-0c8de93b-08e0-419c-a285-afbbfb688c26 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:32:53.246726 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-0c8de93b-08e0-419c-a285-afbbfb688c26 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:32:53.274766 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.objects.instance [None req-0c8de93b-08e0-419c-a285-afbbfb688c26 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lazy-loading 'flavor' on Instance uuid fff5c233-ad7f-4022-8582-e8620f32ebd5 {{(pid=120864) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Aug 27 02:32:53.617400 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-0c8de93b-08e0-419c-a285-afbbfb688c26 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] 10.222.0.52 "POST /compute/v2.1/servers/fff5c233-ad7f-4022-8582-e8620f32ebd5/action" status: 202 len: 0 microversion: 2.20 time: 0.388541 Aug 27 02:32:53.618235 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 467/928] 10.222.0.52 () {62 vars in 1319 bytes} [Sat Aug 27 02:32:53 2022] POST /compute/v2.1/servers/fff5c233-ad7f-4022-8582-e8620f32ebd5/action => generated 0 bytes in 391 msecs (HTTP/1.1 202) 9 headers in 363 bytes (1 switches on core 0) Aug 27 02:32:53.628446 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-6a7a79cb-d0ea-4156-8e3f-4663d502ce90 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:32:53.628988 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-6a7a79cb-d0ea-4156-8e3f-4663d502ce90 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] [instance: fff5c233-ad7f-4022-8582-e8620f32ebd5] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:32:53.636636 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-6a7a79cb-d0ea-4156-8e3f-4663d502ce90 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:32:53.637335 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-6a7a79cb-d0ea-4156-8e3f-4663d502ce90 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:32:53.637823 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-6a7a79cb-d0ea-4156-8e3f-4663d502ce90 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:32:53.638652 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-6a7a79cb-d0ea-4156-8e3f-4663d502ce90 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:32:53.639109 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-6a7a79cb-d0ea-4156-8e3f-4663d502ce90 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:32:53.639543 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-6a7a79cb-d0ea-4156-8e3f-4663d502ce90 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:32:53.895611 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-6a7a79cb-d0ea-4156-8e3f-4663d502ce90 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:32:53.904684 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-6a7a79cb-d0ea-4156-8e3f-4663d502ce90 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:32:53.906135 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-6a7a79cb-d0ea-4156-8e3f-4663d502ce90 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:32:53.907315 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-6a7a79cb-d0ea-4156-8e3f-4663d502ce90 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] 10.222.0.52 "GET /compute/v2.1/servers/fff5c233-ad7f-4022-8582-e8620f32ebd5" status: 200 len: 1725 microversion: 2.20 time: 0.282924 Aug 27 02:32:53.907869 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 462/929] 10.222.0.52 () {60 vars in 1277 bytes} [Sat Aug 27 02:32:53 2022] GET /compute/v2.1/servers/fff5c233-ad7f-4022-8582-e8620f32ebd5 => generated 1725 bytes in 284 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 27 02:32:54.931241 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-7e416775-fb85-4318-916b-046daaf0d3c6 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Calling method '>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:32:54.932031 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-7e416775-fb85-4318-916b-046daaf0d3c6 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] [instance: fff5c233-ad7f-4022-8582-e8620f32ebd5] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:32:54.946711 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-7e416775-fb85-4318-916b-046daaf0d3c6 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:32:54.956526 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-7e416775-fb85-4318-916b-046daaf0d3c6 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:32:54.957021 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-7e416775-fb85-4318-916b-046daaf0d3c6 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.009s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:32:54.957814 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-7e416775-fb85-4318-916b-046daaf0d3c6 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:32:54.958276 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-7e416775-fb85-4318-916b-046daaf0d3c6 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:32:54.958703 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-7e416775-fb85-4318-916b-046daaf0d3c6 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:32:55.099776 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-7e416775-fb85-4318-916b-046daaf0d3c6 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:32:55.108762 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-7e416775-fb85-4318-916b-046daaf0d3c6 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:32:55.110368 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-7e416775-fb85-4318-916b-046daaf0d3c6 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:32:55.111634 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-7e416775-fb85-4318-916b-046daaf0d3c6 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] 10.222.0.52 "GET /compute/v2.1/servers/fff5c233-ad7f-4022-8582-e8620f32ebd5" status: 200 len: 1725 microversion: 2.20 time: 0.185377 Aug 27 02:32:55.112183 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 468/930] 10.222.0.52 () {60 vars in 1277 bytes} [Sat Aug 27 02:32:54 2022] GET /compute/v2.1/servers/fff5c233-ad7f-4022-8582-e8620f32ebd5 => generated 1725 bytes in 187 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 27 02:32:56.134677 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-ee97eee6-3273-44d3-b2f0-b0bc4c45d1e0 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:32:56.135311 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-ee97eee6-3273-44d3-b2f0-b0bc4c45d1e0 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] [instance: fff5c233-ad7f-4022-8582-e8620f32ebd5] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:32:56.142165 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-ee97eee6-3273-44d3-b2f0-b0bc4c45d1e0 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:32:56.142922 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-ee97eee6-3273-44d3-b2f0-b0bc4c45d1e0 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:32:56.143172 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-ee97eee6-3273-44d3-b2f0-b0bc4c45d1e0 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:32:56.143877 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-ee97eee6-3273-44d3-b2f0-b0bc4c45d1e0 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:32:56.144356 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-ee97eee6-3273-44d3-b2f0-b0bc4c45d1e0 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:32:56.144825 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-ee97eee6-3273-44d3-b2f0-b0bc4c45d1e0 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:32:56.284073 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-ee97eee6-3273-44d3-b2f0-b0bc4c45d1e0 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:32:56.292829 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-ee97eee6-3273-44d3-b2f0-b0bc4c45d1e0 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:32:56.294311 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-ee97eee6-3273-44d3-b2f0-b0bc4c45d1e0 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:32:56.295477 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-ee97eee6-3273-44d3-b2f0-b0bc4c45d1e0 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] 10.222.0.52 "GET /compute/v2.1/servers/fff5c233-ad7f-4022-8582-e8620f32ebd5" status: 200 len: 1725 microversion: 2.20 time: 0.164734 Aug 27 02:32:56.296033 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 463/931] 10.222.0.52 () {60 vars in 1277 bytes} [Sat Aug 27 02:32:56 2022] GET /compute/v2.1/servers/fff5c233-ad7f-4022-8582-e8620f32ebd5 => generated 1725 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 27 02:32:57.320376 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-9af2aebe-ca63-4002-b496-42741e7150f6 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Calling method '>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:32:57.321600 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-9af2aebe-ca63-4002-b496-42741e7150f6 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] [instance: fff5c233-ad7f-4022-8582-e8620f32ebd5] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:32:57.330595 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-9af2aebe-ca63-4002-b496-42741e7150f6 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:32:57.330984 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-9af2aebe-ca63-4002-b496-42741e7150f6 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:32:57.331456 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-9af2aebe-ca63-4002-b496-42741e7150f6 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:32:57.332225 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-9af2aebe-ca63-4002-b496-42741e7150f6 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:32:57.332667 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-9af2aebe-ca63-4002-b496-42741e7150f6 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:32:57.333090 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-9af2aebe-ca63-4002-b496-42741e7150f6 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:32:57.485872 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-9af2aebe-ca63-4002-b496-42741e7150f6 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:32:57.494418 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-9af2aebe-ca63-4002-b496-42741e7150f6 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:32:57.495854 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-9af2aebe-ca63-4002-b496-42741e7150f6 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:32:57.497006 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-9af2aebe-ca63-4002-b496-42741e7150f6 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] 10.222.0.52 "GET /compute/v2.1/servers/fff5c233-ad7f-4022-8582-e8620f32ebd5" status: 200 len: 1725 microversion: 2.20 time: 0.181494 Aug 27 02:32:57.497587 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 469/932] 10.222.0.52 () {60 vars in 1277 bytes} [Sat Aug 27 02:32:57 2022] GET /compute/v2.1/servers/fff5c233-ad7f-4022-8582-e8620f32ebd5 => generated 1725 bytes in 183 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 27 02:32:58.527952 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-f2ad15e9-9a3e-45fa-8ec0-17954db14956 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:32:58.529112 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-f2ad15e9-9a3e-45fa-8ec0-17954db14956 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] [instance: fff5c233-ad7f-4022-8582-e8620f32ebd5] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:32:58.538213 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-f2ad15e9-9a3e-45fa-8ec0-17954db14956 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:32:58.538618 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-f2ad15e9-9a3e-45fa-8ec0-17954db14956 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:32:58.539087 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-f2ad15e9-9a3e-45fa-8ec0-17954db14956 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:32:58.539804 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-f2ad15e9-9a3e-45fa-8ec0-17954db14956 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:32:58.540255 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-f2ad15e9-9a3e-45fa-8ec0-17954db14956 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:32:58.540682 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-f2ad15e9-9a3e-45fa-8ec0-17954db14956 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:32:58.708310 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-f2ad15e9-9a3e-45fa-8ec0-17954db14956 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:32:58.720528 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-f2ad15e9-9a3e-45fa-8ec0-17954db14956 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:32:58.722466 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-f2ad15e9-9a3e-45fa-8ec0-17954db14956 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:32:58.723963 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-f2ad15e9-9a3e-45fa-8ec0-17954db14956 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] 10.222.0.52 "GET /compute/v2.1/servers/fff5c233-ad7f-4022-8582-e8620f32ebd5" status: 200 len: 1725 microversion: 2.20 time: 0.200935 Aug 27 02:32:58.724998 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 464/933] 10.222.0.52 () {60 vars in 1277 bytes} [Sat Aug 27 02:32:58 2022] GET /compute/v2.1/servers/fff5c233-ad7f-4022-8582-e8620f32ebd5 => generated 1725 bytes in 203 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 27 02:32:59.752482 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-39730a18-7fb3-4298-a570-0e67605454f2 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Calling method '>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:32:59.753768 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-39730a18-7fb3-4298-a570-0e67605454f2 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] [instance: fff5c233-ad7f-4022-8582-e8620f32ebd5] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:32:59.764034 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-39730a18-7fb3-4298-a570-0e67605454f2 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:32:59.764715 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-39730a18-7fb3-4298-a570-0e67605454f2 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:32:59.765208 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-39730a18-7fb3-4298-a570-0e67605454f2 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:32:59.766344 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-39730a18-7fb3-4298-a570-0e67605454f2 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:32:59.767094 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-39730a18-7fb3-4298-a570-0e67605454f2 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:32:59.767542 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-39730a18-7fb3-4298-a570-0e67605454f2 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:32:59.926274 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-39730a18-7fb3-4298-a570-0e67605454f2 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:32:59.934667 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-39730a18-7fb3-4298-a570-0e67605454f2 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:32:59.936113 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-39730a18-7fb3-4298-a570-0e67605454f2 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:32:59.937360 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-39730a18-7fb3-4298-a570-0e67605454f2 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] 10.222.0.52 "GET /compute/v2.1/servers/fff5c233-ad7f-4022-8582-e8620f32ebd5" status: 200 len: 1725 microversion: 2.20 time: 0.190973 Aug 27 02:32:59.937955 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 470/934] 10.222.0.52 () {60 vars in 1277 bytes} [Sat Aug 27 02:32:59 2022] GET /compute/v2.1/servers/fff5c233-ad7f-4022-8582-e8620f32ebd5 => generated 1725 bytes in 193 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 27 02:33:00.965989 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-07a20181-afe3-458d-a40f-92467c3ce8f0 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:33:00.966840 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-07a20181-afe3-458d-a40f-92467c3ce8f0 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] [instance: fff5c233-ad7f-4022-8582-e8620f32ebd5] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:33:00.976231 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-07a20181-afe3-458d-a40f-92467c3ce8f0 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:33:00.976903 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-07a20181-afe3-458d-a40f-92467c3ce8f0 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:33:00.977656 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-07a20181-afe3-458d-a40f-92467c3ce8f0 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:33:00.978307 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-07a20181-afe3-458d-a40f-92467c3ce8f0 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:33:00.978938 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-07a20181-afe3-458d-a40f-92467c3ce8f0 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:33:00.979527 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-07a20181-afe3-458d-a40f-92467c3ce8f0 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:33:01.135848 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-07a20181-afe3-458d-a40f-92467c3ce8f0 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:33:01.146188 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-07a20181-afe3-458d-a40f-92467c3ce8f0 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:33:01.148127 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-07a20181-afe3-458d-a40f-92467c3ce8f0 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:33:01.149395 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-07a20181-afe3-458d-a40f-92467c3ce8f0 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] 10.222.0.52 "GET /compute/v2.1/servers/fff5c233-ad7f-4022-8582-e8620f32ebd5" status: 200 len: 1725 microversion: 2.20 time: 0.189251 Aug 27 02:33:01.152397 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 465/935] 10.222.0.52 () {60 vars in 1277 bytes} [Sat Aug 27 02:33:00 2022] GET /compute/v2.1/servers/fff5c233-ad7f-4022-8582-e8620f32ebd5 => generated 1725 bytes in 190 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 27 02:33:02.175658 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-8c43fe01-77d5-4178-a5c4-78fbb0de5b93 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Calling method '>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:33:02.176319 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-8c43fe01-77d5-4178-a5c4-78fbb0de5b93 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] [instance: fff5c233-ad7f-4022-8582-e8620f32ebd5] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:33:02.183070 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-8c43fe01-77d5-4178-a5c4-78fbb0de5b93 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:33:02.183550 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-8c43fe01-77d5-4178-a5c4-78fbb0de5b93 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:33:02.183982 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-8c43fe01-77d5-4178-a5c4-78fbb0de5b93 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:33:02.184664 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-8c43fe01-77d5-4178-a5c4-78fbb0de5b93 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:33:02.185111 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-8c43fe01-77d5-4178-a5c4-78fbb0de5b93 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:33:02.185546 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-8c43fe01-77d5-4178-a5c4-78fbb0de5b93 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:33:02.338008 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-8c43fe01-77d5-4178-a5c4-78fbb0de5b93 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:33:02.347434 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-8c43fe01-77d5-4178-a5c4-78fbb0de5b93 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:33:02.348865 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-8c43fe01-77d5-4178-a5c4-78fbb0de5b93 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:33:02.350070 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-8c43fe01-77d5-4178-a5c4-78fbb0de5b93 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] 10.222.0.52 "GET /compute/v2.1/servers/fff5c233-ad7f-4022-8582-e8620f32ebd5" status: 200 len: 1725 microversion: 2.20 time: 0.178589 Aug 27 02:33:02.350634 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 471/936] 10.222.0.52 () {60 vars in 1277 bytes} [Sat Aug 27 02:33:02 2022] GET /compute/v2.1/servers/fff5c233-ad7f-4022-8582-e8620f32ebd5 => generated 1725 bytes in 180 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Aug 27 02:33:03.375073 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-18aa6367-d4a1-475e-bbe9-a0d5414d119b tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:33:03.377143 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-18aa6367-d4a1-475e-bbe9-a0d5414d119b tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] [instance: fff5c233-ad7f-4022-8582-e8620f32ebd5] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:33:03.388359 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-18aa6367-d4a1-475e-bbe9-a0d5414d119b tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:33:03.389397 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-18aa6367-d4a1-475e-bbe9-a0d5414d119b tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:33:03.390283 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-18aa6367-d4a1-475e-bbe9-a0d5414d119b tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:33:03.391494 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-18aa6367-d4a1-475e-bbe9-a0d5414d119b tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:33:03.392444 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-18aa6367-d4a1-475e-bbe9-a0d5414d119b tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:33:03.393334 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-18aa6367-d4a1-475e-bbe9-a0d5414d119b tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:33:03.556004 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-18aa6367-d4a1-475e-bbe9-a0d5414d119b tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:33:03.564542 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-18aa6367-d4a1-475e-bbe9-a0d5414d119b tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:33:03.566748 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-18aa6367-d4a1-475e-bbe9-a0d5414d119b tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:33:03.567933 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-18aa6367-d4a1-475e-bbe9-a0d5414d119b tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] 10.222.0.52 "GET /compute/v2.1/servers/fff5c233-ad7f-4022-8582-e8620f32ebd5" status: 200 len: 1725 microversion: 2.20 time: 0.197272 Aug 27 02:33:03.568527 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 466/937] 10.222.0.52 () {60 vars in 1277 bytes} [Sat Aug 27 02:33:03 2022] GET /compute/v2.1/servers/fff5c233-ad7f-4022-8582-e8620f32ebd5 => generated 1725 bytes in 199 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 27 02:33:04.591950 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-40f690cf-2b7d-458e-aa81-2e05f6199f3e tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Calling method '>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:33:04.592908 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-40f690cf-2b7d-458e-aa81-2e05f6199f3e tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] [instance: fff5c233-ad7f-4022-8582-e8620f32ebd5] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:33:04.601491 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-40f690cf-2b7d-458e-aa81-2e05f6199f3e tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:33:04.602129 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-40f690cf-2b7d-458e-aa81-2e05f6199f3e tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:33:04.602660 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-40f690cf-2b7d-458e-aa81-2e05f6199f3e tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:33:04.603462 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-40f690cf-2b7d-458e-aa81-2e05f6199f3e tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:33:04.603936 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-40f690cf-2b7d-458e-aa81-2e05f6199f3e tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:33:04.604370 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-40f690cf-2b7d-458e-aa81-2e05f6199f3e tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:33:04.752668 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-40f690cf-2b7d-458e-aa81-2e05f6199f3e tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:33:04.761933 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-40f690cf-2b7d-458e-aa81-2e05f6199f3e tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:33:04.763365 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-40f690cf-2b7d-458e-aa81-2e05f6199f3e tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:33:04.764488 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-40f690cf-2b7d-458e-aa81-2e05f6199f3e tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] 10.222.0.52 "GET /compute/v2.1/servers/fff5c233-ad7f-4022-8582-e8620f32ebd5" status: 200 len: 1725 microversion: 2.20 time: 0.176566 Aug 27 02:33:04.765046 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 472/938] 10.222.0.52 () {60 vars in 1277 bytes} [Sat Aug 27 02:33:04 2022] GET /compute/v2.1/servers/fff5c233-ad7f-4022-8582-e8620f32ebd5 => generated 1725 bytes in 178 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 27 02:33:05.788372 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-ad4b750f-06cd-47dd-9dd4-46960008dcdc tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:33:05.789372 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-ad4b750f-06cd-47dd-9dd4-46960008dcdc tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] [instance: fff5c233-ad7f-4022-8582-e8620f32ebd5] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:33:05.795916 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-ad4b750f-06cd-47dd-9dd4-46960008dcdc tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:33:05.796424 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-ad4b750f-06cd-47dd-9dd4-46960008dcdc tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:33:05.796858 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-ad4b750f-06cd-47dd-9dd4-46960008dcdc tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:33:05.797589 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-ad4b750f-06cd-47dd-9dd4-46960008dcdc tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:33:05.798035 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-ad4b750f-06cd-47dd-9dd4-46960008dcdc tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:33:05.798459 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-ad4b750f-06cd-47dd-9dd4-46960008dcdc tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:33:05.941439 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-ad4b750f-06cd-47dd-9dd4-46960008dcdc tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:33:05.950717 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-ad4b750f-06cd-47dd-9dd4-46960008dcdc tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:33:05.952152 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-ad4b750f-06cd-47dd-9dd4-46960008dcdc tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:33:05.953360 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-ad4b750f-06cd-47dd-9dd4-46960008dcdc tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] 10.222.0.52 "GET /compute/v2.1/servers/fff5c233-ad7f-4022-8582-e8620f32ebd5" status: 200 len: 1746 microversion: 2.20 time: 0.169164 Aug 27 02:33:05.953946 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 467/939] 10.222.0.52 () {60 vars in 1277 bytes} [Sat Aug 27 02:33:05 2022] GET /compute/v2.1/servers/fff5c233-ad7f-4022-8582-e8620f32ebd5 => generated 1746 bytes in 170 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 27 02:33:06.977229 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-46e01f66-635d-436b-94bf-2388aeda9fcc tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Calling method '>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:33:06.977779 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-46e01f66-635d-436b-94bf-2388aeda9fcc tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] [instance: fff5c233-ad7f-4022-8582-e8620f32ebd5] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:33:06.984929 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-46e01f66-635d-436b-94bf-2388aeda9fcc tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:33:06.985448 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-46e01f66-635d-436b-94bf-2388aeda9fcc tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:33:06.985880 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-46e01f66-635d-436b-94bf-2388aeda9fcc tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:33:06.986912 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-46e01f66-635d-436b-94bf-2388aeda9fcc tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:33:06.987191 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-46e01f66-635d-436b-94bf-2388aeda9fcc tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:33:06.987608 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-46e01f66-635d-436b-94bf-2388aeda9fcc tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:33:07.137373 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-46e01f66-635d-436b-94bf-2388aeda9fcc tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:33:07.147354 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-46e01f66-635d-436b-94bf-2388aeda9fcc tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:33:07.148797 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-46e01f66-635d-436b-94bf-2388aeda9fcc tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:33:07.150025 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-46e01f66-635d-436b-94bf-2388aeda9fcc tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] 10.222.0.52 "GET /compute/v2.1/servers/fff5c233-ad7f-4022-8582-e8620f32ebd5" status: 200 len: 1746 microversion: 2.20 time: 0.176847 Aug 27 02:33:07.150607 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 473/940] 10.222.0.52 () {60 vars in 1277 bytes} [Sat Aug 27 02:33:06 2022] GET /compute/v2.1/servers/fff5c233-ad7f-4022-8582-e8620f32ebd5 => generated 1746 bytes in 178 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 27 02:33:08.175906 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-46cb25b5-a2eb-46c6-a435-e975cf1c9553 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:33:08.176556 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-46cb25b5-a2eb-46c6-a435-e975cf1c9553 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] [instance: fff5c233-ad7f-4022-8582-e8620f32ebd5] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:33:08.183820 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-46cb25b5-a2eb-46c6-a435-e975cf1c9553 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:33:08.184302 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-46cb25b5-a2eb-46c6-a435-e975cf1c9553 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:33:08.184734 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-46cb25b5-a2eb-46c6-a435-e975cf1c9553 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:33:08.185511 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-46cb25b5-a2eb-46c6-a435-e975cf1c9553 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:33:08.186010 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-46cb25b5-a2eb-46c6-a435-e975cf1c9553 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:33:08.186430 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-46cb25b5-a2eb-46c6-a435-e975cf1c9553 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:33:08.589948 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-46cb25b5-a2eb-46c6-a435-e975cf1c9553 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:33:08.598894 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-46cb25b5-a2eb-46c6-a435-e975cf1c9553 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:33:08.600502 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-46cb25b5-a2eb-46c6-a435-e975cf1c9553 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:33:08.601708 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-46cb25b5-a2eb-46c6-a435-e975cf1c9553 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] 10.222.0.52 "GET /compute/v2.1/servers/fff5c233-ad7f-4022-8582-e8620f32ebd5" status: 200 len: 1746 microversion: 2.20 time: 0.429958 Aug 27 02:33:08.602376 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 468/941] 10.222.0.52 () {60 vars in 1277 bytes} [Sat Aug 27 02:33:08 2022] GET /compute/v2.1/servers/fff5c233-ad7f-4022-8582-e8620f32ebd5 => generated 1746 bytes in 431 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 27 02:33:09.633415 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-a10b8d5f-1687-4989-acea-606f040893c3 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Calling method '>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:33:09.634045 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-a10b8d5f-1687-4989-acea-606f040893c3 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] [instance: fff5c233-ad7f-4022-8582-e8620f32ebd5] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:33:09.640531 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-a10b8d5f-1687-4989-acea-606f040893c3 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:33:09.641018 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-a10b8d5f-1687-4989-acea-606f040893c3 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:33:09.641472 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-a10b8d5f-1687-4989-acea-606f040893c3 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:33:09.642166 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-a10b8d5f-1687-4989-acea-606f040893c3 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:33:09.642612 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-a10b8d5f-1687-4989-acea-606f040893c3 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:33:09.643150 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-a10b8d5f-1687-4989-acea-606f040893c3 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:33:09.785542 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-a10b8d5f-1687-4989-acea-606f040893c3 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:33:09.796433 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-a10b8d5f-1687-4989-acea-606f040893c3 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:33:09.798632 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-a10b8d5f-1687-4989-acea-606f040893c3 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:33:09.799983 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-a10b8d5f-1687-4989-acea-606f040893c3 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] 10.222.0.52 "GET /compute/v2.1/servers/fff5c233-ad7f-4022-8582-e8620f32ebd5" status: 200 len: 1746 microversion: 2.20 time: 0.177837 Aug 27 02:33:09.800537 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 474/942] 10.222.0.52 () {60 vars in 1277 bytes} [Sat Aug 27 02:33:09 2022] GET /compute/v2.1/servers/fff5c233-ad7f-4022-8582-e8620f32ebd5 => generated 1746 bytes in 179 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 27 02:33:10.831828 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-2138cabf-2031-466e-b0f9-7d9fe3a49001 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:33:10.832456 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-2138cabf-2031-466e-b0f9-7d9fe3a49001 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] [instance: fff5c233-ad7f-4022-8582-e8620f32ebd5] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:33:10.839413 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-2138cabf-2031-466e-b0f9-7d9fe3a49001 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:33:10.839904 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-2138cabf-2031-466e-b0f9-7d9fe3a49001 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:33:10.840340 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-2138cabf-2031-466e-b0f9-7d9fe3a49001 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:33:10.841334 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-2138cabf-2031-466e-b0f9-7d9fe3a49001 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:33:10.841792 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-2138cabf-2031-466e-b0f9-7d9fe3a49001 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:33:10.842218 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-2138cabf-2031-466e-b0f9-7d9fe3a49001 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:33:10.985390 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-2138cabf-2031-466e-b0f9-7d9fe3a49001 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:33:10.994718 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-2138cabf-2031-466e-b0f9-7d9fe3a49001 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:33:10.996156 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-2138cabf-2031-466e-b0f9-7d9fe3a49001 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:33:10.997364 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-2138cabf-2031-466e-b0f9-7d9fe3a49001 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] 10.222.0.52 "GET /compute/v2.1/servers/fff5c233-ad7f-4022-8582-e8620f32ebd5" status: 200 len: 1746 microversion: 2.20 time: 0.171456 Aug 27 02:33:10.997925 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 469/943] 10.222.0.52 () {60 vars in 1277 bytes} [Sat Aug 27 02:33:10 2022] GET /compute/v2.1/servers/fff5c233-ad7f-4022-8582-e8620f32ebd5 => generated 1746 bytes in 173 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 27 02:33:11.254162 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [req-a5d6749a-4ee6-4e08-aedc-24602cc19324 req-a6dc3a22-ac33-45c4-b4fc-ad93e3020a1a service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "fff5c233-ad7f-4022-8582-e8620f32ebd5", "name": "network-vif-plugged", "status": "completed", "tag": "67fc5822-1682-4324-ae1c-b98d8d2d09e8"}]} {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 27 02:33:11.264783 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [req-a5d6749a-4ee6-4e08-aedc-24602cc19324 req-a6dc3a22-ac33-45c4-b4fc-ad93e3020a1a service nova] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:33:11.265293 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [req-a5d6749a-4ee6-4e08-aedc-24602cc19324 req-a6dc3a22-ac33-45c4-b4fc-ad93e3020a1a service nova] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:33:11.265768 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [req-a5d6749a-4ee6-4e08-aedc-24602cc19324 req-a6dc3a22-ac33-45c4-b4fc-ad93e3020a1a service nova] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:33:11.279826 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.compute.server_external_events [req-a5d6749a-4ee6-4e08-aedc-24602cc19324 req-a6dc3a22-ac33-45c4-b4fc-ad93e3020a1a service nova] Creating event network-vif-plugged:67fc5822-1682-4324-ae1c-b98d8d2d09e8 for instance fff5c233-ad7f-4022-8582-e8620f32ebd5 on hv-cinder-84630 Aug 27 02:33:11.285486 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [req-a5d6749a-4ee6-4e08-aedc-24602cc19324 req-a6dc3a22-ac33-45c4-b4fc-ad93e3020a1a service nova] 10.222.0.52 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.035116 Aug 27 02:33:11.286277 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 475/944] 10.222.0.52 () {64 vars in 1317 bytes} [Sat Aug 27 02:33:11 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 37 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Aug 27 02:33:12.021421 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-21d78f17-9183-492c-a231-ecb43362b0f6 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:33:12.022063 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-21d78f17-9183-492c-a231-ecb43362b0f6 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] [instance: fff5c233-ad7f-4022-8582-e8620f32ebd5] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:33:12.028850 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-21d78f17-9183-492c-a231-ecb43362b0f6 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:33:12.029408 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-21d78f17-9183-492c-a231-ecb43362b0f6 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:33:12.029880 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-21d78f17-9183-492c-a231-ecb43362b0f6 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:33:12.030583 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-21d78f17-9183-492c-a231-ecb43362b0f6 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:33:12.031014 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-21d78f17-9183-492c-a231-ecb43362b0f6 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:33:12.031464 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-21d78f17-9183-492c-a231-ecb43362b0f6 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:33:12.322485 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-21d78f17-9183-492c-a231-ecb43362b0f6 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:33:12.331151 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-21d78f17-9183-492c-a231-ecb43362b0f6 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:33:12.332639 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-21d78f17-9183-492c-a231-ecb43362b0f6 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:33:12.333853 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-21d78f17-9183-492c-a231-ecb43362b0f6 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] 10.222.0.52 "GET /compute/v2.1/servers/fff5c233-ad7f-4022-8582-e8620f32ebd5" status: 200 len: 1746 microversion: 2.20 time: 0.316760 Aug 27 02:33:12.334441 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 470/945] 10.222.0.52 () {60 vars in 1277 bytes} [Sat Aug 27 02:33:12 2022] GET /compute/v2.1/servers/fff5c233-ad7f-4022-8582-e8620f32ebd5 => generated 1746 bytes in 318 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 27 02:33:13.360192 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-60837d2a-c154-4754-b67b-b0e6be2a358c tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Calling method '>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:33:13.360890 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-60837d2a-c154-4754-b67b-b0e6be2a358c tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] [instance: fff5c233-ad7f-4022-8582-e8620f32ebd5] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:33:13.370207 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-60837d2a-c154-4754-b67b-b0e6be2a358c tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:33:13.370971 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-60837d2a-c154-4754-b67b-b0e6be2a358c tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:33:13.371583 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-60837d2a-c154-4754-b67b-b0e6be2a358c tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:33:13.372321 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-60837d2a-c154-4754-b67b-b0e6be2a358c tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:33:13.372743 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-60837d2a-c154-4754-b67b-b0e6be2a358c tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:33:13.373195 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-60837d2a-c154-4754-b67b-b0e6be2a358c tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:33:13.532717 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-60837d2a-c154-4754-b67b-b0e6be2a358c tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:33:13.542556 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-60837d2a-c154-4754-b67b-b0e6be2a358c tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:33:13.543997 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-60837d2a-c154-4754-b67b-b0e6be2a358c tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:33:13.545168 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-60837d2a-c154-4754-b67b-b0e6be2a358c tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] 10.222.0.52 "GET /compute/v2.1/servers/fff5c233-ad7f-4022-8582-e8620f32ebd5" status: 200 len: 1746 microversion: 2.20 time: 0.192314 Aug 27 02:33:13.545729 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 476/946] 10.222.0.52 () {60 vars in 1277 bytes} [Sat Aug 27 02:33:13 2022] GET /compute/v2.1/servers/fff5c233-ad7f-4022-8582-e8620f32ebd5 => generated 1746 bytes in 194 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 27 02:33:14.571117 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-1e065fd8-0abb-4828-b2c2-e5184208d5c3 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:33:14.572003 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-1e065fd8-0abb-4828-b2c2-e5184208d5c3 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] [instance: fff5c233-ad7f-4022-8582-e8620f32ebd5] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:33:14.581543 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-1e065fd8-0abb-4828-b2c2-e5184208d5c3 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:33:14.582033 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-1e065fd8-0abb-4828-b2c2-e5184208d5c3 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:33:14.582476 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-1e065fd8-0abb-4828-b2c2-e5184208d5c3 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:33:14.583162 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-1e065fd8-0abb-4828-b2c2-e5184208d5c3 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:33:14.583617 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-1e065fd8-0abb-4828-b2c2-e5184208d5c3 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:33:14.584038 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-1e065fd8-0abb-4828-b2c2-e5184208d5c3 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:33:14.722878 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-1e065fd8-0abb-4828-b2c2-e5184208d5c3 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:33:14.735471 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-1e065fd8-0abb-4828-b2c2-e5184208d5c3 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:33:14.737587 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-1e065fd8-0abb-4828-b2c2-e5184208d5c3 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:33:14.738751 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-1e065fd8-0abb-4828-b2c2-e5184208d5c3 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] 10.222.0.52 "GET /compute/v2.1/servers/fff5c233-ad7f-4022-8582-e8620f32ebd5" status: 200 len: 1746 microversion: 2.20 time: 0.173152 Aug 27 02:33:14.739313 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 471/947] 10.222.0.52 () {60 vars in 1277 bytes} [Sat Aug 27 02:33:14 2022] GET /compute/v2.1/servers/fff5c233-ad7f-4022-8582-e8620f32ebd5 => generated 1746 bytes in 175 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 27 02:33:15.762560 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-e4f86885-2c53-4303-9187-217648eec34a tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Calling method '>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:33:15.763124 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-e4f86885-2c53-4303-9187-217648eec34a tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] [instance: fff5c233-ad7f-4022-8582-e8620f32ebd5] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:33:15.770920 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-e4f86885-2c53-4303-9187-217648eec34a tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:33:15.771457 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-e4f86885-2c53-4303-9187-217648eec34a tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:33:15.771943 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-e4f86885-2c53-4303-9187-217648eec34a tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:33:15.772727 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-e4f86885-2c53-4303-9187-217648eec34a tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:33:15.773216 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-e4f86885-2c53-4303-9187-217648eec34a tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:33:15.773662 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-e4f86885-2c53-4303-9187-217648eec34a tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:33:15.914071 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-e4f86885-2c53-4303-9187-217648eec34a tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:33:15.926799 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-e4f86885-2c53-4303-9187-217648eec34a tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:33:15.928231 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-e4f86885-2c53-4303-9187-217648eec34a tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:33:15.929420 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-e4f86885-2c53-4303-9187-217648eec34a tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] 10.222.0.52 "GET /compute/v2.1/servers/fff5c233-ad7f-4022-8582-e8620f32ebd5" status: 200 len: 1746 microversion: 2.20 time: 0.171222 Aug 27 02:33:15.929980 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 477/948] 10.222.0.52 () {60 vars in 1277 bytes} [Sat Aug 27 02:33:15 2022] GET /compute/v2.1/servers/fff5c233-ad7f-4022-8582-e8620f32ebd5 => generated 1746 bytes in 173 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 27 02:33:16.960952 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-7ef00563-4f02-45e7-8632-9e722755f4ee tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:33:16.961613 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-7ef00563-4f02-45e7-8632-9e722755f4ee tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] [instance: fff5c233-ad7f-4022-8582-e8620f32ebd5] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:33:16.968236 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-7ef00563-4f02-45e7-8632-9e722755f4ee tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:33:16.968692 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-7ef00563-4f02-45e7-8632-9e722755f4ee tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:33:16.969140 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-7ef00563-4f02-45e7-8632-9e722755f4ee tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:33:16.969823 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-7ef00563-4f02-45e7-8632-9e722755f4ee tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:33:16.970486 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-7ef00563-4f02-45e7-8632-9e722755f4ee tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:33:16.970902 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-7ef00563-4f02-45e7-8632-9e722755f4ee tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:33:17.108635 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-7ef00563-4f02-45e7-8632-9e722755f4ee tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:33:17.117676 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-7ef00563-4f02-45e7-8632-9e722755f4ee tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:33:17.119394 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-7ef00563-4f02-45e7-8632-9e722755f4ee tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:33:17.120529 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-7ef00563-4f02-45e7-8632-9e722755f4ee tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] 10.222.0.52 "GET /compute/v2.1/servers/fff5c233-ad7f-4022-8582-e8620f32ebd5" status: 200 len: 1741 microversion: 2.20 time: 0.163636 Aug 27 02:33:17.121076 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 472/949] 10.222.0.52 () {60 vars in 1277 bytes} [Sat Aug 27 02:33:16 2022] GET /compute/v2.1/servers/fff5c233-ad7f-4022-8582-e8620f32ebd5 => generated 1741 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 27 02:33:18.144881 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-8369bf1f-617a-4c41-b05d-d3621be0da96 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Calling method '>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:33:18.145545 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-8369bf1f-617a-4c41-b05d-d3621be0da96 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] [instance: fff5c233-ad7f-4022-8582-e8620f32ebd5] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:33:18.152318 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-8369bf1f-617a-4c41-b05d-d3621be0da96 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:33:18.152911 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-8369bf1f-617a-4c41-b05d-d3621be0da96 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:33:18.153437 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-8369bf1f-617a-4c41-b05d-d3621be0da96 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:33:18.154424 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-8369bf1f-617a-4c41-b05d-d3621be0da96 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:33:18.155144 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-8369bf1f-617a-4c41-b05d-d3621be0da96 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:33:18.155809 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-8369bf1f-617a-4c41-b05d-d3621be0da96 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:33:18.304251 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-8369bf1f-617a-4c41-b05d-d3621be0da96 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:33:18.313238 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-8369bf1f-617a-4c41-b05d-d3621be0da96 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:33:18.315148 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-8369bf1f-617a-4c41-b05d-d3621be0da96 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:33:18.317035 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-8369bf1f-617a-4c41-b05d-d3621be0da96 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] 10.222.0.52 "GET /compute/v2.1/servers/fff5c233-ad7f-4022-8582-e8620f32ebd5" status: 200 len: 1741 microversion: 2.20 time: 0.176473 Aug 27 02:33:18.317645 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 478/950] 10.222.0.52 () {60 vars in 1277 bytes} [Sat Aug 27 02:33:18 2022] GET /compute/v2.1/servers/fff5c233-ad7f-4022-8582-e8620f32ebd5 => generated 1741 bytes in 178 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 27 02:33:19.340712 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-9bcede8a-62d5-4df2-b277-b94a18634a9a tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:33:19.341357 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-9bcede8a-62d5-4df2-b277-b94a18634a9a tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] [instance: fff5c233-ad7f-4022-8582-e8620f32ebd5] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:33:19.349652 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-9bcede8a-62d5-4df2-b277-b94a18634a9a tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:33:19.350140 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-9bcede8a-62d5-4df2-b277-b94a18634a9a tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:33:19.350577 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-9bcede8a-62d5-4df2-b277-b94a18634a9a tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:33:19.351275 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-9bcede8a-62d5-4df2-b277-b94a18634a9a tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:33:19.351731 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-9bcede8a-62d5-4df2-b277-b94a18634a9a tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:33:19.352153 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-9bcede8a-62d5-4df2-b277-b94a18634a9a tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:33:19.520274 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-9bcede8a-62d5-4df2-b277-b94a18634a9a tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:33:19.528498 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-9bcede8a-62d5-4df2-b277-b94a18634a9a tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:33:19.529959 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-9bcede8a-62d5-4df2-b277-b94a18634a9a tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:33:19.531122 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-9bcede8a-62d5-4df2-b277-b94a18634a9a tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] 10.222.0.52 "GET /compute/v2.1/servers/fff5c233-ad7f-4022-8582-e8620f32ebd5" status: 200 len: 1741 microversion: 2.20 time: 0.194812 Aug 27 02:33:19.531644 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 473/951] 10.222.0.52 () {60 vars in 1277 bytes} [Sat Aug 27 02:33:19 2022] GET /compute/v2.1/servers/fff5c233-ad7f-4022-8582-e8620f32ebd5 => generated 1741 bytes in 196 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 27 02:33:20.561393 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-c3fd8967-1581-4fda-a3df-e25328b00583 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Calling method '>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:33:20.562068 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-c3fd8967-1581-4fda-a3df-e25328b00583 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] [instance: fff5c233-ad7f-4022-8582-e8620f32ebd5] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:33:20.569817 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-c3fd8967-1581-4fda-a3df-e25328b00583 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:33:20.570469 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-c3fd8967-1581-4fda-a3df-e25328b00583 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:33:20.570904 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-c3fd8967-1581-4fda-a3df-e25328b00583 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:33:20.571626 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-c3fd8967-1581-4fda-a3df-e25328b00583 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:33:20.572072 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-c3fd8967-1581-4fda-a3df-e25328b00583 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:33:20.572481 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-c3fd8967-1581-4fda-a3df-e25328b00583 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:33:20.735280 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-c3fd8967-1581-4fda-a3df-e25328b00583 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:33:20.743787 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-c3fd8967-1581-4fda-a3df-e25328b00583 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:33:20.745283 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-c3fd8967-1581-4fda-a3df-e25328b00583 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:33:20.746428 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-c3fd8967-1581-4fda-a3df-e25328b00583 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] 10.222.0.52 "GET /compute/v2.1/servers/fff5c233-ad7f-4022-8582-e8620f32ebd5" status: 200 len: 1741 microversion: 2.20 time: 0.189551 Aug 27 02:33:20.746980 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 479/952] 10.222.0.52 () {60 vars in 1277 bytes} [Sat Aug 27 02:33:20 2022] GET /compute/v2.1/servers/fff5c233-ad7f-4022-8582-e8620f32ebd5 => generated 1741 bytes in 191 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 27 02:33:21.773369 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-c3dc7efe-9822-41bc-a1b8-0490022edf53 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:33:21.774556 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-c3dc7efe-9822-41bc-a1b8-0490022edf53 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] [instance: fff5c233-ad7f-4022-8582-e8620f32ebd5] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:33:21.784273 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG dbcounter [-] [120863] Writing DB stats nova_api:SELECT=27 {{(pid=120863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 27 02:33:21.788528 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-c3dc7efe-9822-41bc-a1b8-0490022edf53 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:33:21.789766 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-c3dc7efe-9822-41bc-a1b8-0490022edf53 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:33:21.791392 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-c3dc7efe-9822-41bc-a1b8-0490022edf53 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:33:21.792983 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-c3dc7efe-9822-41bc-a1b8-0490022edf53 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:33:21.793628 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-c3dc7efe-9822-41bc-a1b8-0490022edf53 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:33:21.794340 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-c3dc7efe-9822-41bc-a1b8-0490022edf53 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:33:21.804397 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG dbcounter [-] [120863] Writing DB stats nova_cell1:SELECT=77 {{(pid=120863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 27 02:33:21.962610 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-c3dc7efe-9822-41bc-a1b8-0490022edf53 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:33:21.971169 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-c3dc7efe-9822-41bc-a1b8-0490022edf53 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:33:21.973758 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-c3dc7efe-9822-41bc-a1b8-0490022edf53 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:33:21.974749 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-c3dc7efe-9822-41bc-a1b8-0490022edf53 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] 10.222.0.52 "GET /compute/v2.1/servers/fff5c233-ad7f-4022-8582-e8620f32ebd5" status: 200 len: 1741 microversion: 2.20 time: 0.208027 Aug 27 02:33:21.975385 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 474/953] 10.222.0.52 () {60 vars in 1277 bytes} [Sat Aug 27 02:33:21 2022] GET /compute/v2.1/servers/fff5c233-ad7f-4022-8582-e8620f32ebd5 => generated 1741 bytes in 210 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 27 02:33:23.004761 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-e6d6b75f-0488-4303-a220-82c027b278cd tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Calling method '>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:33:23.005398 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-e6d6b75f-0488-4303-a220-82c027b278cd tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] [instance: fff5c233-ad7f-4022-8582-e8620f32ebd5] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:33:23.012221 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-e6d6b75f-0488-4303-a220-82c027b278cd tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:33:23.012705 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-e6d6b75f-0488-4303-a220-82c027b278cd tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:33:23.013191 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-e6d6b75f-0488-4303-a220-82c027b278cd tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:33:23.013900 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-e6d6b75f-0488-4303-a220-82c027b278cd tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:33:23.014350 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-e6d6b75f-0488-4303-a220-82c027b278cd tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:33:23.014771 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-e6d6b75f-0488-4303-a220-82c027b278cd tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:33:23.156976 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-e6d6b75f-0488-4303-a220-82c027b278cd tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:33:23.168861 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-e6d6b75f-0488-4303-a220-82c027b278cd tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:33:23.170401 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-e6d6b75f-0488-4303-a220-82c027b278cd tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:33:23.171558 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-e6d6b75f-0488-4303-a220-82c027b278cd tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] 10.222.0.52 "GET /compute/v2.1/servers/fff5c233-ad7f-4022-8582-e8620f32ebd5" status: 200 len: 1741 microversion: 2.20 time: 0.170826 Aug 27 02:33:23.172105 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 480/954] 10.222.0.52 () {60 vars in 1277 bytes} [Sat Aug 27 02:33:22 2022] GET /compute/v2.1/servers/fff5c233-ad7f-4022-8582-e8620f32ebd5 => generated 1741 bytes in 172 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 27 02:33:24.195489 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-519d8869-235c-4c77-bea6-33695a6e65e7 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:33:24.196205 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-519d8869-235c-4c77-bea6-33695a6e65e7 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] [instance: fff5c233-ad7f-4022-8582-e8620f32ebd5] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:33:24.203187 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-519d8869-235c-4c77-bea6-33695a6e65e7 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:33:24.203673 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-519d8869-235c-4c77-bea6-33695a6e65e7 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:33:24.204132 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-519d8869-235c-4c77-bea6-33695a6e65e7 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:33:24.204817 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-519d8869-235c-4c77-bea6-33695a6e65e7 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:33:24.205290 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-519d8869-235c-4c77-bea6-33695a6e65e7 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:33:24.205712 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-519d8869-235c-4c77-bea6-33695a6e65e7 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:33:24.348169 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-519d8869-235c-4c77-bea6-33695a6e65e7 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:33:24.356724 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-519d8869-235c-4c77-bea6-33695a6e65e7 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:33:24.358216 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-519d8869-235c-4c77-bea6-33695a6e65e7 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:33:24.359675 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-519d8869-235c-4c77-bea6-33695a6e65e7 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] 10.222.0.52 "GET /compute/v2.1/servers/fff5c233-ad7f-4022-8582-e8620f32ebd5" status: 200 len: 1741 microversion: 2.20 time: 0.168299 Aug 27 02:33:24.360311 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 475/955] 10.222.0.52 () {60 vars in 1277 bytes} [Sat Aug 27 02:33:24 2022] GET /compute/v2.1/servers/fff5c233-ad7f-4022-8582-e8620f32ebd5 => generated 1741 bytes in 170 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Aug 27 02:33:25.383437 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-dac917a7-a912-4b22-8229-c9afed51b69f tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Calling method '>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:33:25.384063 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-dac917a7-a912-4b22-8229-c9afed51b69f tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] [instance: fff5c233-ad7f-4022-8582-e8620f32ebd5] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:33:25.386564 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG dbcounter [-] [120864] Writing DB stats nova_api:SELECT=29 {{(pid=120864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 27 02:33:25.392432 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-dac917a7-a912-4b22-8229-c9afed51b69f tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:33:25.392908 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-dac917a7-a912-4b22-8229-c9afed51b69f tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:33:25.393356 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-dac917a7-a912-4b22-8229-c9afed51b69f tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:33:25.394123 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-dac917a7-a912-4b22-8229-c9afed51b69f tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:33:25.394655 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-dac917a7-a912-4b22-8229-c9afed51b69f tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:33:25.395097 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-dac917a7-a912-4b22-8229-c9afed51b69f tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:33:25.397790 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG dbcounter [-] [120864] Writing DB stats nova_cell1:SELECT=90,nova_cell1:UPDATE=2,nova_cell1:INSERT=1 {{(pid=120864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 27 02:33:25.537515 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-dac917a7-a912-4b22-8229-c9afed51b69f tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:33:25.546600 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-dac917a7-a912-4b22-8229-c9afed51b69f tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:33:25.548039 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-dac917a7-a912-4b22-8229-c9afed51b69f tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:33:25.549296 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-dac917a7-a912-4b22-8229-c9afed51b69f tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] 10.222.0.52 "GET /compute/v2.1/servers/fff5c233-ad7f-4022-8582-e8620f32ebd5" status: 200 len: 1741 microversion: 2.20 time: 0.169759 Aug 27 02:33:25.550082 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 481/956] 10.222.0.52 () {60 vars in 1277 bytes} [Sat Aug 27 02:33:25 2022] GET /compute/v2.1/servers/fff5c233-ad7f-4022-8582-e8620f32ebd5 => generated 1741 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 27 02:33:26.576021 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-4fb60ca4-d19a-42f1-bbab-74ac6413209e tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:33:26.576660 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-4fb60ca4-d19a-42f1-bbab-74ac6413209e tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] [instance: fff5c233-ad7f-4022-8582-e8620f32ebd5] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:33:26.584789 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-4fb60ca4-d19a-42f1-bbab-74ac6413209e tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:33:26.586127 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-4fb60ca4-d19a-42f1-bbab-74ac6413209e tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:33:26.586127 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-4fb60ca4-d19a-42f1-bbab-74ac6413209e tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:33:26.586652 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-4fb60ca4-d19a-42f1-bbab-74ac6413209e tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:33:26.587114 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-4fb60ca4-d19a-42f1-bbab-74ac6413209e tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:33:26.587567 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-4fb60ca4-d19a-42f1-bbab-74ac6413209e tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:33:26.731770 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-4fb60ca4-d19a-42f1-bbab-74ac6413209e tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:33:26.741547 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-4fb60ca4-d19a-42f1-bbab-74ac6413209e tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:33:26.743252 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-4fb60ca4-d19a-42f1-bbab-74ac6413209e tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:33:26.744714 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-4fb60ca4-d19a-42f1-bbab-74ac6413209e tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] 10.222.0.52 "GET /compute/v2.1/servers/fff5c233-ad7f-4022-8582-e8620f32ebd5" status: 200 len: 1741 microversion: 2.20 time: 0.175011 Aug 27 02:33:26.745264 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 476/957] 10.222.0.52 () {60 vars in 1277 bytes} [Sat Aug 27 02:33:26 2022] GET /compute/v2.1/servers/fff5c233-ad7f-4022-8582-e8620f32ebd5 => generated 1741 bytes in 176 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 27 02:33:27.768345 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-96c29641-e6ac-43f4-9022-571c6359e5cd tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Calling method '>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:33:27.769093 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-96c29641-e6ac-43f4-9022-571c6359e5cd tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] [instance: fff5c233-ad7f-4022-8582-e8620f32ebd5] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:33:27.776407 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-96c29641-e6ac-43f4-9022-571c6359e5cd tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:33:27.776892 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-96c29641-e6ac-43f4-9022-571c6359e5cd tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:33:27.777499 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-96c29641-e6ac-43f4-9022-571c6359e5cd tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:33:27.778224 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-96c29641-e6ac-43f4-9022-571c6359e5cd tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:33:27.778671 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-96c29641-e6ac-43f4-9022-571c6359e5cd tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:33:27.779120 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-96c29641-e6ac-43f4-9022-571c6359e5cd tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:33:27.920277 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-96c29641-e6ac-43f4-9022-571c6359e5cd tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:33:27.929491 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-96c29641-e6ac-43f4-9022-571c6359e5cd tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:33:27.931134 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-96c29641-e6ac-43f4-9022-571c6359e5cd tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:33:27.932495 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-96c29641-e6ac-43f4-9022-571c6359e5cd tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] 10.222.0.52 "GET /compute/v2.1/servers/fff5c233-ad7f-4022-8582-e8620f32ebd5" status: 200 len: 1741 microversion: 2.20 time: 0.168431 Aug 27 02:33:27.933266 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 482/958] 10.222.0.52 () {60 vars in 1277 bytes} [Sat Aug 27 02:33:27 2022] GET /compute/v2.1/servers/fff5c233-ad7f-4022-8582-e8620f32ebd5 => generated 1741 bytes in 170 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 27 02:33:28.957949 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-ea86e06c-d7ce-47db-97cb-d4dd88661d29 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:33:28.958566 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-ea86e06c-d7ce-47db-97cb-d4dd88661d29 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] [instance: fff5c233-ad7f-4022-8582-e8620f32ebd5] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:33:28.966923 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-ea86e06c-d7ce-47db-97cb-d4dd88661d29 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:33:28.967500 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-ea86e06c-d7ce-47db-97cb-d4dd88661d29 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:33:28.968321 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-ea86e06c-d7ce-47db-97cb-d4dd88661d29 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:33:28.969449 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-ea86e06c-d7ce-47db-97cb-d4dd88661d29 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:33:28.970152 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-ea86e06c-d7ce-47db-97cb-d4dd88661d29 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:33:28.970839 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-ea86e06c-d7ce-47db-97cb-d4dd88661d29 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:33:29.127863 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-ea86e06c-d7ce-47db-97cb-d4dd88661d29 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:33:29.137330 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-ea86e06c-d7ce-47db-97cb-d4dd88661d29 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:33:29.139064 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-ea86e06c-d7ce-47db-97cb-d4dd88661d29 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:33:29.140510 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-ea86e06c-d7ce-47db-97cb-d4dd88661d29 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] 10.222.0.52 "GET /compute/v2.1/servers/fff5c233-ad7f-4022-8582-e8620f32ebd5" status: 200 len: 1741 microversion: 2.20 time: 0.187994 Aug 27 02:33:29.141504 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 477/959] 10.222.0.52 () {60 vars in 1277 bytes} [Sat Aug 27 02:33:28 2022] GET /compute/v2.1/servers/fff5c233-ad7f-4022-8582-e8620f32ebd5 => generated 1741 bytes in 189 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 27 02:33:30.165416 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-a4f77f04-007d-4a3e-b783-8d0395b11192 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Calling method '>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:33:30.166314 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-a4f77f04-007d-4a3e-b783-8d0395b11192 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] [instance: fff5c233-ad7f-4022-8582-e8620f32ebd5] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:33:30.172879 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-a4f77f04-007d-4a3e-b783-8d0395b11192 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:33:30.173401 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-a4f77f04-007d-4a3e-b783-8d0395b11192 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:33:30.173834 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-a4f77f04-007d-4a3e-b783-8d0395b11192 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:33:30.174576 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-a4f77f04-007d-4a3e-b783-8d0395b11192 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:33:30.175029 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-a4f77f04-007d-4a3e-b783-8d0395b11192 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:33:30.175617 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-a4f77f04-007d-4a3e-b783-8d0395b11192 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:33:30.350483 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-a4f77f04-007d-4a3e-b783-8d0395b11192 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:33:30.358961 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-a4f77f04-007d-4a3e-b783-8d0395b11192 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:33:30.360472 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-a4f77f04-007d-4a3e-b783-8d0395b11192 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:33:30.361589 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-a4f77f04-007d-4a3e-b783-8d0395b11192 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] 10.222.0.52 "GET /compute/v2.1/servers/fff5c233-ad7f-4022-8582-e8620f32ebd5" status: 200 len: 1741 microversion: 2.20 time: 0.200298 Aug 27 02:33:30.377005 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 483/960] 10.222.0.52 () {60 vars in 1277 bytes} [Sat Aug 27 02:33:30 2022] GET /compute/v2.1/servers/fff5c233-ad7f-4022-8582-e8620f32ebd5 => generated 1741 bytes in 217 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 27 02:33:31.384893 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-d77c49b7-e45b-49a0-b543-2f6492172405 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:33:31.385581 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-d77c49b7-e45b-49a0-b543-2f6492172405 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] [instance: fff5c233-ad7f-4022-8582-e8620f32ebd5] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:33:31.392255 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-d77c49b7-e45b-49a0-b543-2f6492172405 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:33:31.392750 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-d77c49b7-e45b-49a0-b543-2f6492172405 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:33:31.393234 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-d77c49b7-e45b-49a0-b543-2f6492172405 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:33:31.393967 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-d77c49b7-e45b-49a0-b543-2f6492172405 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:33:31.394516 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-d77c49b7-e45b-49a0-b543-2f6492172405 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:33:31.394992 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-d77c49b7-e45b-49a0-b543-2f6492172405 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:33:31.541455 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-d77c49b7-e45b-49a0-b543-2f6492172405 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:33:31.550288 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-d77c49b7-e45b-49a0-b543-2f6492172405 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:33:31.552448 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-d77c49b7-e45b-49a0-b543-2f6492172405 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:33:31.553597 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-d77c49b7-e45b-49a0-b543-2f6492172405 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] 10.222.0.52 "GET /compute/v2.1/servers/fff5c233-ad7f-4022-8582-e8620f32ebd5" status: 200 len: 1741 microversion: 2.20 time: 0.172918 Aug 27 02:33:31.554197 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 478/961] 10.222.0.52 () {60 vars in 1277 bytes} [Sat Aug 27 02:33:31 2022] GET /compute/v2.1/servers/fff5c233-ad7f-4022-8582-e8620f32ebd5 => generated 1741 bytes in 174 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 27 02:33:32.589107 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-2622a1f6-3908-4b9e-b6ba-66eb29fc57ce tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Calling method '>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:33:32.590250 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-2622a1f6-3908-4b9e-b6ba-66eb29fc57ce tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] [instance: fff5c233-ad7f-4022-8582-e8620f32ebd5] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:33:32.599126 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-2622a1f6-3908-4b9e-b6ba-66eb29fc57ce tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:33:32.599627 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-2622a1f6-3908-4b9e-b6ba-66eb29fc57ce tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:33:32.600054 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-2622a1f6-3908-4b9e-b6ba-66eb29fc57ce tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:33:32.600747 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-2622a1f6-3908-4b9e-b6ba-66eb29fc57ce tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:33:32.601389 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-2622a1f6-3908-4b9e-b6ba-66eb29fc57ce tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:33:32.602480 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-2622a1f6-3908-4b9e-b6ba-66eb29fc57ce tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:33:32.751264 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-2622a1f6-3908-4b9e-b6ba-66eb29fc57ce tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:33:32.760751 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-2622a1f6-3908-4b9e-b6ba-66eb29fc57ce tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:33:32.762242 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-2622a1f6-3908-4b9e-b6ba-66eb29fc57ce tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:33:32.763391 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-2622a1f6-3908-4b9e-b6ba-66eb29fc57ce tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] 10.222.0.52 "GET /compute/v2.1/servers/fff5c233-ad7f-4022-8582-e8620f32ebd5" status: 200 len: 1741 microversion: 2.20 time: 0.180832 Aug 27 02:33:32.763961 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 484/962] 10.222.0.52 () {60 vars in 1277 bytes} [Sat Aug 27 02:33:32 2022] GET /compute/v2.1/servers/fff5c233-ad7f-4022-8582-e8620f32ebd5 => generated 1741 bytes in 182 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 27 02:33:33.795675 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-afd26d1b-65ec-43fc-a1ef-3154f2fe0ddd tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:33:33.796465 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-afd26d1b-65ec-43fc-a1ef-3154f2fe0ddd tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] [instance: fff5c233-ad7f-4022-8582-e8620f32ebd5] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:33:33.804078 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-afd26d1b-65ec-43fc-a1ef-3154f2fe0ddd tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:33:33.804603 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-afd26d1b-65ec-43fc-a1ef-3154f2fe0ddd tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:33:33.805027 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-afd26d1b-65ec-43fc-a1ef-3154f2fe0ddd tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:33:33.805788 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-afd26d1b-65ec-43fc-a1ef-3154f2fe0ddd tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:33:33.806240 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-afd26d1b-65ec-43fc-a1ef-3154f2fe0ddd tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:33:33.806664 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-afd26d1b-65ec-43fc-a1ef-3154f2fe0ddd tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:33:33.965434 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-afd26d1b-65ec-43fc-a1ef-3154f2fe0ddd tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:33:33.974947 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-afd26d1b-65ec-43fc-a1ef-3154f2fe0ddd tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:33:33.976624 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-afd26d1b-65ec-43fc-a1ef-3154f2fe0ddd tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:33:33.978883 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-afd26d1b-65ec-43fc-a1ef-3154f2fe0ddd tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] 10.222.0.52 "GET /compute/v2.1/servers/fff5c233-ad7f-4022-8582-e8620f32ebd5" status: 200 len: 1741 microversion: 2.20 time: 0.187834 Aug 27 02:33:33.979699 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 479/963] 10.222.0.52 () {60 vars in 1277 bytes} [Sat Aug 27 02:33:33 2022] GET /compute/v2.1/servers/fff5c233-ad7f-4022-8582-e8620f32ebd5 => generated 1741 bytes in 190 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 27 02:33:35.004497 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-6d60147b-17fb-4042-b5bb-de8a9699a1f5 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Calling method '>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:33:35.005068 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-6d60147b-17fb-4042-b5bb-de8a9699a1f5 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] [instance: fff5c233-ad7f-4022-8582-e8620f32ebd5] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:33:35.011973 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-6d60147b-17fb-4042-b5bb-de8a9699a1f5 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:33:35.012490 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-6d60147b-17fb-4042-b5bb-de8a9699a1f5 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:33:35.012916 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-6d60147b-17fb-4042-b5bb-de8a9699a1f5 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:33:35.013610 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-6d60147b-17fb-4042-b5bb-de8a9699a1f5 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:33:35.014081 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-6d60147b-17fb-4042-b5bb-de8a9699a1f5 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:33:35.014496 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-6d60147b-17fb-4042-b5bb-de8a9699a1f5 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:33:35.161001 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-6d60147b-17fb-4042-b5bb-de8a9699a1f5 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:33:35.169629 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-6d60147b-17fb-4042-b5bb-de8a9699a1f5 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:33:35.171064 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-6d60147b-17fb-4042-b5bb-de8a9699a1f5 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:33:35.172222 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-6d60147b-17fb-4042-b5bb-de8a9699a1f5 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] 10.222.0.52 "GET /compute/v2.1/servers/fff5c233-ad7f-4022-8582-e8620f32ebd5" status: 200 len: 1723 microversion: 2.20 time: 0.172291 Aug 27 02:33:35.172769 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 485/964] 10.222.0.52 () {60 vars in 1277 bytes} [Sat Aug 27 02:33:34 2022] GET /compute/v2.1/servers/fff5c233-ad7f-4022-8582-e8620f32ebd5 => generated 1723 bytes in 173 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 27 02:33:36.198719 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-ee2e652b-3c57-4954-ac90-0f9ec5ec3aa9 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:33:36.199372 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-ee2e652b-3c57-4954-ac90-0f9ec5ec3aa9 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] [instance: fff5c233-ad7f-4022-8582-e8620f32ebd5] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:33:36.208758 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-ee2e652b-3c57-4954-ac90-0f9ec5ec3aa9 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:33:36.209368 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-ee2e652b-3c57-4954-ac90-0f9ec5ec3aa9 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:33:36.209820 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-ee2e652b-3c57-4954-ac90-0f9ec5ec3aa9 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:33:36.210568 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-ee2e652b-3c57-4954-ac90-0f9ec5ec3aa9 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:33:36.211017 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-ee2e652b-3c57-4954-ac90-0f9ec5ec3aa9 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:33:36.211609 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-ee2e652b-3c57-4954-ac90-0f9ec5ec3aa9 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:33:36.307762 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [req-32dd3ef5-003a-4084-9282-9d117b4b0899 req-33c6a0fe-b1da-480a-b497-a2a367e77ad4 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "fff5c233-ad7f-4022-8582-e8620f32ebd5", "name": "network-vif-unplugged", "status": "completed", "tag": "67fc5822-1682-4324-ae1c-b98d8d2d09e8"}]} {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 27 02:33:36.318627 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [req-32dd3ef5-003a-4084-9282-9d117b4b0899 req-33c6a0fe-b1da-480a-b497-a2a367e77ad4 service nova] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:33:36.319361 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [req-32dd3ef5-003a-4084-9282-9d117b4b0899 req-33c6a0fe-b1da-480a-b497-a2a367e77ad4 service nova] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:33:36.319788 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [req-32dd3ef5-003a-4084-9282-9d117b4b0899 req-33c6a0fe-b1da-480a-b497-a2a367e77ad4 service nova] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:33:36.339780 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.compute.server_external_events [req-32dd3ef5-003a-4084-9282-9d117b4b0899 req-33c6a0fe-b1da-480a-b497-a2a367e77ad4 service nova] Creating event network-vif-unplugged:67fc5822-1682-4324-ae1c-b98d8d2d09e8 for instance fff5c233-ad7f-4022-8582-e8620f32ebd5 on hv-cinder-84630 Aug 27 02:33:36.348360 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [req-32dd3ef5-003a-4084-9282-9d117b4b0899 req-33c6a0fe-b1da-480a-b497-a2a367e77ad4 service nova] 10.222.0.52 "POST /compute/v2.1/os-server-external-events" status: 200 len: 185 microversion: 2.1 time: 0.044086 Aug 27 02:33:36.350154 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 486/965] 10.222.0.52 () {64 vars in 1317 bytes} [Sat Aug 27 02:33:36 2022] POST /compute/v2.1/os-server-external-events => generated 185 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 27 02:33:36.395287 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-ee2e652b-3c57-4954-ac90-0f9ec5ec3aa9 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:33:36.404596 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-ee2e652b-3c57-4954-ac90-0f9ec5ec3aa9 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:33:36.406293 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-ee2e652b-3c57-4954-ac90-0f9ec5ec3aa9 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:33:36.407552 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-ee2e652b-3c57-4954-ac90-0f9ec5ec3aa9 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] 10.222.0.52 "GET /compute/v2.1/servers/fff5c233-ad7f-4022-8582-e8620f32ebd5" status: 200 len: 1723 microversion: 2.20 time: 0.214459 Aug 27 02:33:36.408255 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 480/966] 10.222.0.52 () {60 vars in 1277 bytes} [Sat Aug 27 02:33:36 2022] GET /compute/v2.1/servers/fff5c233-ad7f-4022-8582-e8620f32ebd5 => generated 1723 bytes in 216 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 27 02:33:37.432401 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-4f41a582-80f7-4219-9a14-8ec1f48ff32c tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Calling method '>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:33:37.433034 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-4f41a582-80f7-4219-9a14-8ec1f48ff32c tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] [instance: fff5c233-ad7f-4022-8582-e8620f32ebd5] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:33:37.440039 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-4f41a582-80f7-4219-9a14-8ec1f48ff32c tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:33:37.440526 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-4f41a582-80f7-4219-9a14-8ec1f48ff32c tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:33:37.440957 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-4f41a582-80f7-4219-9a14-8ec1f48ff32c tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:33:37.441899 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-4f41a582-80f7-4219-9a14-8ec1f48ff32c tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:33:37.442350 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-4f41a582-80f7-4219-9a14-8ec1f48ff32c tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:33:37.442772 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-4f41a582-80f7-4219-9a14-8ec1f48ff32c tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:33:37.581051 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-4f41a582-80f7-4219-9a14-8ec1f48ff32c tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:33:37.589557 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-4f41a582-80f7-4219-9a14-8ec1f48ff32c tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:33:37.591088 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-4f41a582-80f7-4219-9a14-8ec1f48ff32c tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:33:37.592316 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-4f41a582-80f7-4219-9a14-8ec1f48ff32c tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] 10.222.0.52 "GET /compute/v2.1/servers/fff5c233-ad7f-4022-8582-e8620f32ebd5" status: 200 len: 1723 microversion: 2.20 time: 0.164730 Aug 27 02:33:37.593063 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 487/967] 10.222.0.52 () {60 vars in 1277 bytes} [Sat Aug 27 02:33:37 2022] GET /compute/v2.1/servers/fff5c233-ad7f-4022-8582-e8620f32ebd5 => generated 1723 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 27 02:33:38.616753 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-3b4435b7-234d-491d-9978-c741cb8d9a99 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:33:38.618161 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-3b4435b7-234d-491d-9978-c741cb8d9a99 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] [instance: fff5c233-ad7f-4022-8582-e8620f32ebd5] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:33:38.626097 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-3b4435b7-234d-491d-9978-c741cb8d9a99 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:33:38.626650 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-3b4435b7-234d-491d-9978-c741cb8d9a99 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:33:38.627295 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-3b4435b7-234d-491d-9978-c741cb8d9a99 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:33:38.627991 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-3b4435b7-234d-491d-9978-c741cb8d9a99 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:33:38.628605 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-3b4435b7-234d-491d-9978-c741cb8d9a99 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:33:38.629967 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-3b4435b7-234d-491d-9978-c741cb8d9a99 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:33:38.782490 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-3b4435b7-234d-491d-9978-c741cb8d9a99 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:33:38.792740 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-3b4435b7-234d-491d-9978-c741cb8d9a99 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:33:38.794467 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-3b4435b7-234d-491d-9978-c741cb8d9a99 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:33:38.795655 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-3b4435b7-234d-491d-9978-c741cb8d9a99 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] 10.222.0.52 "GET /compute/v2.1/servers/fff5c233-ad7f-4022-8582-e8620f32ebd5" status: 200 len: 1726 microversion: 2.20 time: 0.183407 Aug 27 02:33:38.797632 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 481/968] 10.222.0.52 () {60 vars in 1277 bytes} [Sat Aug 27 02:33:38 2022] GET /compute/v2.1/servers/fff5c233-ad7f-4022-8582-e8620f32ebd5 => generated 1726 bytes in 186 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 27 02:33:38.820678 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-9b773ad2-ddc3-4bb8-a299-3ee4f166f160 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "2b96366f-76ff-4842-87d6-0c393b0b4bb4"}} {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 27 02:33:38.823169 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-9b773ad2-ddc3-4bb8-a299-3ee4f166f160 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] [instance: fff5c233-ad7f-4022-8582-e8620f32ebd5] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:33:38.831511 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-9b773ad2-ddc3-4bb8-a299-3ee4f166f160 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:33:38.831511 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-9b773ad2-ddc3-4bb8-a299-3ee4f166f160 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:33:38.831511 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-9b773ad2-ddc3-4bb8-a299-3ee4f166f160 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:33:38.831511 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-9b773ad2-ddc3-4bb8-a299-3ee4f166f160 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:33:38.831958 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-9b773ad2-ddc3-4bb8-a299-3ee4f166f160 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:33:38.832564 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-9b773ad2-ddc3-4bb8-a299-3ee4f166f160 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:33:39.144464 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-9b773ad2-ddc3-4bb8-a299-3ee4f166f160 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] 10.222.0.52 "POST /compute/v2.1/servers/fff5c233-ad7f-4022-8582-e8620f32ebd5/os-volume_attachments" status: 200 len: 188 microversion: 2.20 time: 0.327895 Aug 27 02:33:39.145481 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 488/969] 10.222.0.52 () {62 vars in 1364 bytes} [Sat Aug 27 02:33:38 2022] POST /compute/v2.1/servers/fff5c233-ad7f-4022-8582-e8620f32ebd5/os-volume_attachments => generated 188 bytes in 330 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Aug 27 02:33:39.274721 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-cb1b2334-b4b9-44f3-b25b-e461c9cb181f tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:33:39.275215 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-cb1b2334-b4b9-44f3-b25b-e461c9cb181f tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] [instance: fff5c233-ad7f-4022-8582-e8620f32ebd5] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:33:39.281275 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-cb1b2334-b4b9-44f3-b25b-e461c9cb181f tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:33:39.282251 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-cb1b2334-b4b9-44f3-b25b-e461c9cb181f tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:33:39.282700 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-cb1b2334-b4b9-44f3-b25b-e461c9cb181f tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:33:39.283403 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-cb1b2334-b4b9-44f3-b25b-e461c9cb181f tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:33:39.283859 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-cb1b2334-b4b9-44f3-b25b-e461c9cb181f tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:33:39.284272 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-cb1b2334-b4b9-44f3-b25b-e461c9cb181f tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:33:39.568123 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-cb1b2334-b4b9-44f3-b25b-e461c9cb181f tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] 10.222.0.52 "DELETE /compute/v2.1/servers/fff5c233-ad7f-4022-8582-e8620f32ebd5/os-volume_attachments/2b96366f-76ff-4842-87d6-0c393b0b4bb4" status: 202 len: 0 microversion: 2.20 time: 0.297442 Aug 27 02:33:39.568825 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 482/970] 10.222.0.52 () {60 vars in 1457 bytes} [Sat Aug 27 02:33:39 2022] DELETE /compute/v2.1/servers/fff5c233-ad7f-4022-8582-e8620f32ebd5/os-volume_attachments/2b96366f-76ff-4842-87d6-0c393b0b4bb4 => generated 0 bytes in 299 msecs (HTTP/1.1 202) 9 headers in 363 bytes (1 switches on core 0) Aug 27 02:33:39.630092 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-3e792dbc-98aa-4827-b4ed-797303141a46 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Action: 'action', calling method: >, body: {"unshelve": null} {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 27 02:33:39.630594 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-3e792dbc-98aa-4827-b4ed-797303141a46 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] [instance: fff5c233-ad7f-4022-8582-e8620f32ebd5] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:33:39.638148 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-3e792dbc-98aa-4827-b4ed-797303141a46 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:33:39.638622 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-3e792dbc-98aa-4827-b4ed-797303141a46 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:33:39.639050 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-3e792dbc-98aa-4827-b4ed-797303141a46 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:33:39.639734 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-3e792dbc-98aa-4827-b4ed-797303141a46 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:33:39.640179 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-3e792dbc-98aa-4827-b4ed-797303141a46 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:33:39.640594 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-3e792dbc-98aa-4827-b4ed-797303141a46 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:33:39.671564 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-3e792dbc-98aa-4827-b4ed-797303141a46 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] [instance: fff5c233-ad7f-4022-8582-e8620f32ebd5] Unshelving instance with old availability_zone "None" to new availability_zone "not provided" and host "None". {{(pid=120864) unshelve /opt/stack/nova/nova/compute/api.py:4591}} Aug 27 02:33:39.742879 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR oslo.messaging._drivers.impl_rabbit [None req-3e792dbc-98aa-4827-b4ed-797303141a46 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] [d65ce105-b0fd-41c0-9d27-8f13ed1b64da] AMQP server on 10.222.0.52:5672 is unreachable: [Errno 104] Connection reset by peer. Trying again in 1 seconds.: ConnectionResetError: [Errno 104] Connection reset by peer Aug 27 02:33:40.752051 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO oslo.messaging._drivers.impl_rabbit [None req-3e792dbc-98aa-4827-b4ed-797303141a46 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] [d65ce105-b0fd-41c0-9d27-8f13ed1b64da] Reconnected to AMQP server on 10.222.0.52:5672 via [amqp] client with port 47754. Aug 27 02:33:40.756617 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-3e792dbc-98aa-4827-b4ed-797303141a46 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] 10.222.0.52 "POST /compute/v2.1/servers/fff5c233-ad7f-4022-8582-e8620f32ebd5/action" status: 202 len: 0 microversion: 2.20 time: 1.130985 Aug 27 02:33:40.757319 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 489/971] 10.222.0.52 () {62 vars in 1319 bytes} [Sat Aug 27 02:33:39 2022] POST /compute/v2.1/servers/fff5c233-ad7f-4022-8582-e8620f32ebd5/action => generated 0 bytes in 1132 msecs (HTTP/1.1 202) 9 headers in 363 bytes (1 switches on core 0) Aug 27 02:33:40.765804 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-369bcaea-8b48-4799-bfff-9a820b6e20ee tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:33:40.766408 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-369bcaea-8b48-4799-bfff-9a820b6e20ee tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] [instance: fff5c233-ad7f-4022-8582-e8620f32ebd5] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:33:40.773983 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-369bcaea-8b48-4799-bfff-9a820b6e20ee tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:33:40.774461 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-369bcaea-8b48-4799-bfff-9a820b6e20ee tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:33:40.774892 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-369bcaea-8b48-4799-bfff-9a820b6e20ee tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:33:40.775582 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-369bcaea-8b48-4799-bfff-9a820b6e20ee tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:33:40.776019 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-369bcaea-8b48-4799-bfff-9a820b6e20ee tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:33:40.776438 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-369bcaea-8b48-4799-bfff-9a820b6e20ee tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:33:40.912058 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-369bcaea-8b48-4799-bfff-9a820b6e20ee tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:33:40.920298 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-369bcaea-8b48-4799-bfff-9a820b6e20ee tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:33:40.921746 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-369bcaea-8b48-4799-bfff-9a820b6e20ee tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:33:40.922889 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-369bcaea-8b48-4799-bfff-9a820b6e20ee tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] 10.222.0.52 "GET /compute/v2.1/servers/fff5c233-ad7f-4022-8582-e8620f32ebd5" status: 200 len: 1674 microversion: 2.20 time: 0.161132 Aug 27 02:33:40.923417 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 483/972] 10.222.0.52 () {60 vars in 1277 bytes} [Sat Aug 27 02:33:40 2022] GET /compute/v2.1/servers/fff5c233-ad7f-4022-8582-e8620f32ebd5 => generated 1674 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 27 02:33:41.947396 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-a723c362-352e-4889-ab5c-ccbe862d2ba0 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Calling method '>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:33:41.948017 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-a723c362-352e-4889-ab5c-ccbe862d2ba0 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] [instance: fff5c233-ad7f-4022-8582-e8620f32ebd5] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:33:41.955248 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-a723c362-352e-4889-ab5c-ccbe862d2ba0 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:33:41.955733 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-a723c362-352e-4889-ab5c-ccbe862d2ba0 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:33:41.956165 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-a723c362-352e-4889-ab5c-ccbe862d2ba0 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:33:41.956868 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-a723c362-352e-4889-ab5c-ccbe862d2ba0 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:33:41.957333 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-a723c362-352e-4889-ab5c-ccbe862d2ba0 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:33:41.957752 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-a723c362-352e-4889-ab5c-ccbe862d2ba0 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:33:42.134970 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-a723c362-352e-4889-ab5c-ccbe862d2ba0 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:33:42.143952 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-a723c362-352e-4889-ab5c-ccbe862d2ba0 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:33:42.145354 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-a723c362-352e-4889-ab5c-ccbe862d2ba0 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:33:42.146949 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-a723c362-352e-4889-ab5c-ccbe862d2ba0 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] 10.222.0.52 "GET /compute/v2.1/servers/fff5c233-ad7f-4022-8582-e8620f32ebd5" status: 200 len: 1674 microversion: 2.20 time: 0.204231 Aug 27 02:33:42.147484 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 490/973] 10.222.0.52 () {60 vars in 1277 bytes} [Sat Aug 27 02:33:41 2022] GET /compute/v2.1/servers/fff5c233-ad7f-4022-8582-e8620f32ebd5 => generated 1674 bytes in 206 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 27 02:33:43.171690 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-47854c55-2602-485e-81ef-849d06112431 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:33:43.172333 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-47854c55-2602-485e-81ef-849d06112431 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] [instance: fff5c233-ad7f-4022-8582-e8620f32ebd5] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:33:43.179551 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-47854c55-2602-485e-81ef-849d06112431 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:33:43.180053 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-47854c55-2602-485e-81ef-849d06112431 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:33:43.180489 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-47854c55-2602-485e-81ef-849d06112431 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:33:43.181259 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-47854c55-2602-485e-81ef-849d06112431 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:33:43.181766 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-47854c55-2602-485e-81ef-849d06112431 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:33:43.182489 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-47854c55-2602-485e-81ef-849d06112431 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:33:43.333622 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-47854c55-2602-485e-81ef-849d06112431 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:33:43.342267 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-47854c55-2602-485e-81ef-849d06112431 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:33:43.343695 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-47854c55-2602-485e-81ef-849d06112431 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:33:43.344866 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-47854c55-2602-485e-81ef-849d06112431 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] 10.222.0.52 "GET /compute/v2.1/servers/fff5c233-ad7f-4022-8582-e8620f32ebd5" status: 200 len: 1674 microversion: 2.20 time: 0.177259 Aug 27 02:33:43.345468 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 484/974] 10.222.0.52 () {60 vars in 1277 bytes} [Sat Aug 27 02:33:43 2022] GET /compute/v2.1/servers/fff5c233-ad7f-4022-8582-e8620f32ebd5 => generated 1674 bytes in 178 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 27 02:33:44.372300 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-93529d46-1df6-43e6-89bf-409bd5eb7acb tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Calling method '>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:33:44.373245 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-93529d46-1df6-43e6-89bf-409bd5eb7acb tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] [instance: fff5c233-ad7f-4022-8582-e8620f32ebd5] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:33:44.380684 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-93529d46-1df6-43e6-89bf-409bd5eb7acb tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:33:44.381166 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-93529d46-1df6-43e6-89bf-409bd5eb7acb tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:33:44.381613 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-93529d46-1df6-43e6-89bf-409bd5eb7acb tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:33:44.382328 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-93529d46-1df6-43e6-89bf-409bd5eb7acb tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:33:44.382768 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-93529d46-1df6-43e6-89bf-409bd5eb7acb tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:33:44.383175 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-93529d46-1df6-43e6-89bf-409bd5eb7acb tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:33:44.532074 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-93529d46-1df6-43e6-89bf-409bd5eb7acb tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:33:44.541587 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-93529d46-1df6-43e6-89bf-409bd5eb7acb tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:33:44.543127 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-93529d46-1df6-43e6-89bf-409bd5eb7acb tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:33:44.544361 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-93529d46-1df6-43e6-89bf-409bd5eb7acb tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] 10.222.0.52 "GET /compute/v2.1/servers/fff5c233-ad7f-4022-8582-e8620f32ebd5" status: 200 len: 1674 microversion: 2.20 time: 0.177927 Aug 27 02:33:44.544998 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 491/975] 10.222.0.52 () {60 vars in 1277 bytes} [Sat Aug 27 02:33:44 2022] GET /compute/v2.1/servers/fff5c233-ad7f-4022-8582-e8620f32ebd5 => generated 1674 bytes in 180 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 27 02:33:45.572041 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-48def009-9efa-458d-b0f4-c23d9d3e3f41 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:33:45.573875 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-48def009-9efa-458d-b0f4-c23d9d3e3f41 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] [instance: fff5c233-ad7f-4022-8582-e8620f32ebd5] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:33:45.582452 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-48def009-9efa-458d-b0f4-c23d9d3e3f41 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:33:45.583143 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-48def009-9efa-458d-b0f4-c23d9d3e3f41 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:33:45.583883 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-48def009-9efa-458d-b0f4-c23d9d3e3f41 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:33:45.584685 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-48def009-9efa-458d-b0f4-c23d9d3e3f41 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:33:45.585100 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-48def009-9efa-458d-b0f4-c23d9d3e3f41 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:33:45.585525 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-48def009-9efa-458d-b0f4-c23d9d3e3f41 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:33:45.729718 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-48def009-9efa-458d-b0f4-c23d9d3e3f41 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:33:45.741487 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-48def009-9efa-458d-b0f4-c23d9d3e3f41 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:33:45.742950 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-48def009-9efa-458d-b0f4-c23d9d3e3f41 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:33:45.744145 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-48def009-9efa-458d-b0f4-c23d9d3e3f41 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] 10.222.0.52 "GET /compute/v2.1/servers/fff5c233-ad7f-4022-8582-e8620f32ebd5" status: 200 len: 1674 microversion: 2.20 time: 0.177748 Aug 27 02:33:45.744683 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 485/976] 10.222.0.52 () {60 vars in 1277 bytes} [Sat Aug 27 02:33:45 2022] GET /compute/v2.1/servers/fff5c233-ad7f-4022-8582-e8620f32ebd5 => generated 1674 bytes in 179 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 27 02:33:46.774134 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-a99c91ec-272b-4383-911e-4edd7d9ba8a9 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Calling method '>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:33:46.775131 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-a99c91ec-272b-4383-911e-4edd7d9ba8a9 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] [instance: fff5c233-ad7f-4022-8582-e8620f32ebd5] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:33:46.782693 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-a99c91ec-272b-4383-911e-4edd7d9ba8a9 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:33:46.783202 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-a99c91ec-272b-4383-911e-4edd7d9ba8a9 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:33:46.783627 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-a99c91ec-272b-4383-911e-4edd7d9ba8a9 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:33:46.784312 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-a99c91ec-272b-4383-911e-4edd7d9ba8a9 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:33:46.784755 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-a99c91ec-272b-4383-911e-4edd7d9ba8a9 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:33:46.785171 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-a99c91ec-272b-4383-911e-4edd7d9ba8a9 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:33:46.929372 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-a99c91ec-272b-4383-911e-4edd7d9ba8a9 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:33:46.937646 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-a99c91ec-272b-4383-911e-4edd7d9ba8a9 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:33:46.939106 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-a99c91ec-272b-4383-911e-4edd7d9ba8a9 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:33:46.940265 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-a99c91ec-272b-4383-911e-4edd7d9ba8a9 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] 10.222.0.52 "GET /compute/v2.1/servers/fff5c233-ad7f-4022-8582-e8620f32ebd5" status: 200 len: 1732 microversion: 2.20 time: 0.172928 Aug 27 02:33:46.940823 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 492/977] 10.222.0.52 () {60 vars in 1277 bytes} [Sat Aug 27 02:33:46 2022] GET /compute/v2.1/servers/fff5c233-ad7f-4022-8582-e8620f32ebd5 => generated 1732 bytes in 174 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 27 02:33:47.964725 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-9f363968-7843-4960-8cf5-13e489d8cc0c tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:33:47.965602 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-9f363968-7843-4960-8cf5-13e489d8cc0c tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] [instance: fff5c233-ad7f-4022-8582-e8620f32ebd5] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:33:47.971999 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-9f363968-7843-4960-8cf5-13e489d8cc0c tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:33:47.972408 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-9f363968-7843-4960-8cf5-13e489d8cc0c tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:33:47.973176 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-9f363968-7843-4960-8cf5-13e489d8cc0c tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:33:47.974102 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-9f363968-7843-4960-8cf5-13e489d8cc0c tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:33:47.974562 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-9f363968-7843-4960-8cf5-13e489d8cc0c tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:33:47.974960 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-9f363968-7843-4960-8cf5-13e489d8cc0c tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:33:48.121542 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-9f363968-7843-4960-8cf5-13e489d8cc0c tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:33:48.129946 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-9f363968-7843-4960-8cf5-13e489d8cc0c tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:33:48.131490 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-9f363968-7843-4960-8cf5-13e489d8cc0c tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:33:48.132906 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-9f363968-7843-4960-8cf5-13e489d8cc0c tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] 10.222.0.52 "GET /compute/v2.1/servers/fff5c233-ad7f-4022-8582-e8620f32ebd5" status: 200 len: 1732 microversion: 2.20 time: 0.172371 Aug 27 02:33:48.133687 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 486/978] 10.222.0.52 () {60 vars in 1277 bytes} [Sat Aug 27 02:33:47 2022] GET /compute/v2.1/servers/fff5c233-ad7f-4022-8582-e8620f32ebd5 => generated 1732 bytes in 174 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 27 02:33:49.160494 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-1ab0bc09-dc5d-4e49-9a61-9116fdb8b29e tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Calling method '>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:33:49.161493 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-1ab0bc09-dc5d-4e49-9a61-9116fdb8b29e tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] [instance: fff5c233-ad7f-4022-8582-e8620f32ebd5] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:33:49.168443 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-1ab0bc09-dc5d-4e49-9a61-9116fdb8b29e tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:33:49.168919 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-1ab0bc09-dc5d-4e49-9a61-9116fdb8b29e tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:33:49.169381 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-1ab0bc09-dc5d-4e49-9a61-9116fdb8b29e tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:33:49.170064 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-1ab0bc09-dc5d-4e49-9a61-9116fdb8b29e tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:33:49.170512 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-1ab0bc09-dc5d-4e49-9a61-9116fdb8b29e tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:33:49.170927 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-1ab0bc09-dc5d-4e49-9a61-9116fdb8b29e tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:33:49.319783 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-1ab0bc09-dc5d-4e49-9a61-9116fdb8b29e tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:33:49.327979 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-1ab0bc09-dc5d-4e49-9a61-9116fdb8b29e tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:33:49.329462 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-1ab0bc09-dc5d-4e49-9a61-9116fdb8b29e tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:33:49.330600 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-1ab0bc09-dc5d-4e49-9a61-9116fdb8b29e tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] 10.222.0.52 "GET /compute/v2.1/servers/fff5c233-ad7f-4022-8582-e8620f32ebd5" status: 200 len: 1732 microversion: 2.20 time: 0.176503 Aug 27 02:33:49.331327 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 493/979] 10.222.0.52 () {60 vars in 1277 bytes} [Sat Aug 27 02:33:49 2022] GET /compute/v2.1/servers/fff5c233-ad7f-4022-8582-e8620f32ebd5 => generated 1732 bytes in 178 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 27 02:33:50.354752 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-dd4117b2-b27f-4fbe-b883-28a3c431d2d0 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:33:50.355368 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-dd4117b2-b27f-4fbe-b883-28a3c431d2d0 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] [instance: fff5c233-ad7f-4022-8582-e8620f32ebd5] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:33:50.362491 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-dd4117b2-b27f-4fbe-b883-28a3c431d2d0 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:33:50.363105 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-dd4117b2-b27f-4fbe-b883-28a3c431d2d0 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:33:50.363615 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-dd4117b2-b27f-4fbe-b883-28a3c431d2d0 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:33:50.364426 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-dd4117b2-b27f-4fbe-b883-28a3c431d2d0 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:33:50.365052 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-dd4117b2-b27f-4fbe-b883-28a3c431d2d0 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:33:50.366456 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-dd4117b2-b27f-4fbe-b883-28a3c431d2d0 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:33:50.504082 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-dd4117b2-b27f-4fbe-b883-28a3c431d2d0 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:33:50.513250 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-dd4117b2-b27f-4fbe-b883-28a3c431d2d0 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:33:50.514689 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-dd4117b2-b27f-4fbe-b883-28a3c431d2d0 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:33:50.515864 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-dd4117b2-b27f-4fbe-b883-28a3c431d2d0 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] 10.222.0.52 "GET /compute/v2.1/servers/fff5c233-ad7f-4022-8582-e8620f32ebd5" status: 200 len: 1732 microversion: 2.20 time: 0.165059 Aug 27 02:33:50.516415 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 487/980] 10.222.0.52 () {60 vars in 1277 bytes} [Sat Aug 27 02:33:50 2022] GET /compute/v2.1/servers/fff5c233-ad7f-4022-8582-e8620f32ebd5 => generated 1732 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 27 02:33:51.539992 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-8f3b23fb-f6ec-416d-8ad8-956279d00023 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Calling method '>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:33:51.540569 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-8f3b23fb-f6ec-416d-8ad8-956279d00023 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] [instance: fff5c233-ad7f-4022-8582-e8620f32ebd5] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:33:51.547540 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-8f3b23fb-f6ec-416d-8ad8-956279d00023 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:33:51.548031 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-8f3b23fb-f6ec-416d-8ad8-956279d00023 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:33:51.548493 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-8f3b23fb-f6ec-416d-8ad8-956279d00023 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:33:51.549226 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-8f3b23fb-f6ec-416d-8ad8-956279d00023 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:33:51.549646 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-8f3b23fb-f6ec-416d-8ad8-956279d00023 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:33:51.550086 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-8f3b23fb-f6ec-416d-8ad8-956279d00023 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:33:51.693743 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-8f3b23fb-f6ec-416d-8ad8-956279d00023 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:33:51.696717 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG dbcounter [-] [120864] Writing DB stats nova_cell1:SELECT=91,nova_cell1:INSERT=4,nova_cell1:UPDATE=5 {{(pid=120864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 27 02:33:51.705644 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-8f3b23fb-f6ec-416d-8ad8-956279d00023 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:33:51.707112 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-8f3b23fb-f6ec-416d-8ad8-956279d00023 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:33:51.708278 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-8f3b23fb-f6ec-416d-8ad8-956279d00023 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] 10.222.0.52 "GET /compute/v2.1/servers/fff5c233-ad7f-4022-8582-e8620f32ebd5" status: 200 len: 1732 microversion: 2.20 time: 0.172574 Aug 27 02:33:51.709134 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 494/981] 10.222.0.52 () {60 vars in 1277 bytes} [Sat Aug 27 02:33:51 2022] GET /compute/v2.1/servers/fff5c233-ad7f-4022-8582-e8620f32ebd5 => generated 1732 bytes in 174 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 27 02:33:52.736981 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-df9a0688-b702-48e6-a4a2-57b3122cc1c2 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:33:52.738290 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-df9a0688-b702-48e6-a4a2-57b3122cc1c2 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] [instance: fff5c233-ad7f-4022-8582-e8620f32ebd5] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:33:52.741286 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG dbcounter [-] [120863] Writing DB stats nova_api:SELECT=28 {{(pid=120863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 27 02:33:52.747399 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-df9a0688-b702-48e6-a4a2-57b3122cc1c2 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:33:52.748135 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-df9a0688-b702-48e6-a4a2-57b3122cc1c2 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:33:52.748616 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-df9a0688-b702-48e6-a4a2-57b3122cc1c2 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:33:52.749467 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-df9a0688-b702-48e6-a4a2-57b3122cc1c2 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:33:52.750167 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-df9a0688-b702-48e6-a4a2-57b3122cc1c2 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:33:52.750678 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-df9a0688-b702-48e6-a4a2-57b3122cc1c2 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:33:52.755615 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG dbcounter [-] [120863] Writing DB stats nova_cell1:SELECT=93,nova_cell1:INSERT=2,nova_cell1:UPDATE=4 {{(pid=120863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 27 02:33:52.933918 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-df9a0688-b702-48e6-a4a2-57b3122cc1c2 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:33:52.944578 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-df9a0688-b702-48e6-a4a2-57b3122cc1c2 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:33:52.946222 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-df9a0688-b702-48e6-a4a2-57b3122cc1c2 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:33:52.947436 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-df9a0688-b702-48e6-a4a2-57b3122cc1c2 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] 10.222.0.52 "GET /compute/v2.1/servers/fff5c233-ad7f-4022-8582-e8620f32ebd5" status: 200 len: 1732 microversion: 2.20 time: 0.218915 Aug 27 02:33:52.947999 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 488/982] 10.222.0.52 () {60 vars in 1277 bytes} [Sat Aug 27 02:33:52 2022] GET /compute/v2.1/servers/fff5c233-ad7f-4022-8582-e8620f32ebd5 => generated 1732 bytes in 220 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 27 02:33:53.972780 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-c1b72c2c-7a92-4606-be96-5736279d4823 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Calling method '>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:33:53.974695 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-c1b72c2c-7a92-4606-be96-5736279d4823 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] [instance: fff5c233-ad7f-4022-8582-e8620f32ebd5] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:33:53.981606 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-c1b72c2c-7a92-4606-be96-5736279d4823 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:33:53.982091 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-c1b72c2c-7a92-4606-be96-5736279d4823 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:33:53.982521 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-c1b72c2c-7a92-4606-be96-5736279d4823 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:33:53.983226 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-c1b72c2c-7a92-4606-be96-5736279d4823 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:33:53.983895 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-c1b72c2c-7a92-4606-be96-5736279d4823 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:33:53.984449 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-c1b72c2c-7a92-4606-be96-5736279d4823 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:33:54.137201 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-c1b72c2c-7a92-4606-be96-5736279d4823 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:33:54.146804 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-c1b72c2c-7a92-4606-be96-5736279d4823 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:33:54.148198 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-c1b72c2c-7a92-4606-be96-5736279d4823 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:33:54.149434 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-c1b72c2c-7a92-4606-be96-5736279d4823 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] 10.222.0.52 "GET /compute/v2.1/servers/fff5c233-ad7f-4022-8582-e8620f32ebd5" status: 200 len: 1732 microversion: 2.20 time: 0.181941 Aug 27 02:33:54.150444 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 495/983] 10.222.0.52 () {60 vars in 1277 bytes} [Sat Aug 27 02:33:53 2022] GET /compute/v2.1/servers/fff5c233-ad7f-4022-8582-e8620f32ebd5 => generated 1732 bytes in 183 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 27 02:33:55.175596 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-d400a9fb-eebc-43aa-b58d-10f37921750e tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:33:55.176340 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-d400a9fb-eebc-43aa-b58d-10f37921750e tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] [instance: fff5c233-ad7f-4022-8582-e8620f32ebd5] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:33:55.186728 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-d400a9fb-eebc-43aa-b58d-10f37921750e tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:33:55.187433 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-d400a9fb-eebc-43aa-b58d-10f37921750e tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:33:55.187930 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-d400a9fb-eebc-43aa-b58d-10f37921750e tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:33:55.188861 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-d400a9fb-eebc-43aa-b58d-10f37921750e tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:33:55.189528 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-d400a9fb-eebc-43aa-b58d-10f37921750e tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:33:55.189973 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-d400a9fb-eebc-43aa-b58d-10f37921750e tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:33:55.367196 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-d400a9fb-eebc-43aa-b58d-10f37921750e tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:33:55.376178 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-d400a9fb-eebc-43aa-b58d-10f37921750e tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:33:55.377693 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-d400a9fb-eebc-43aa-b58d-10f37921750e tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:33:55.378916 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-d400a9fb-eebc-43aa-b58d-10f37921750e tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] 10.222.0.52 "GET /compute/v2.1/servers/fff5c233-ad7f-4022-8582-e8620f32ebd5" status: 200 len: 1732 microversion: 2.20 time: 0.208652 Aug 27 02:33:55.379758 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 489/984] 10.222.0.52 () {60 vars in 1277 bytes} [Sat Aug 27 02:33:55 2022] GET /compute/v2.1/servers/fff5c233-ad7f-4022-8582-e8620f32ebd5 => generated 1732 bytes in 210 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 27 02:33:56.402651 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-c190da9c-236b-4191-be36-d403d2ff4c95 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Calling method '>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:33:56.403344 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-c190da9c-236b-4191-be36-d403d2ff4c95 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] [instance: fff5c233-ad7f-4022-8582-e8620f32ebd5] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:33:56.405627 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG dbcounter [-] [120864] Writing DB stats nova_api:SELECT=34,nova_api:UPDATE=1 {{(pid=120864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 27 02:33:56.411580 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-c190da9c-236b-4191-be36-d403d2ff4c95 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:33:56.412356 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-c190da9c-236b-4191-be36-d403d2ff4c95 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:33:56.413683 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-c190da9c-236b-4191-be36-d403d2ff4c95 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:33:56.414397 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-c190da9c-236b-4191-be36-d403d2ff4c95 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:33:56.414848 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-c190da9c-236b-4191-be36-d403d2ff4c95 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:33:56.415289 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-c190da9c-236b-4191-be36-d403d2ff4c95 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:33:56.549997 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-c190da9c-236b-4191-be36-d403d2ff4c95 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:33:56.559404 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-c190da9c-236b-4191-be36-d403d2ff4c95 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:33:56.560824 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-c190da9c-236b-4191-be36-d403d2ff4c95 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:33:56.562098 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-c190da9c-236b-4191-be36-d403d2ff4c95 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] 10.222.0.52 "GET /compute/v2.1/servers/fff5c233-ad7f-4022-8582-e8620f32ebd5" status: 200 len: 1732 microversion: 2.20 time: 0.163405 Aug 27 02:33:56.562801 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 496/985] 10.222.0.52 () {60 vars in 1277 bytes} [Sat Aug 27 02:33:56 2022] GET /compute/v2.1/servers/fff5c233-ad7f-4022-8582-e8620f32ebd5 => generated 1732 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 27 02:33:57.586546 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-5b1c5c2e-d00d-4f09-aa84-d83056182d46 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:33:57.587239 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-5b1c5c2e-d00d-4f09-aa84-d83056182d46 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] [instance: fff5c233-ad7f-4022-8582-e8620f32ebd5] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:33:57.594141 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-5b1c5c2e-d00d-4f09-aa84-d83056182d46 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:33:57.594641 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-5b1c5c2e-d00d-4f09-aa84-d83056182d46 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:33:57.595088 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-5b1c5c2e-d00d-4f09-aa84-d83056182d46 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:33:57.595786 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-5b1c5c2e-d00d-4f09-aa84-d83056182d46 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:33:57.596245 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-5b1c5c2e-d00d-4f09-aa84-d83056182d46 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:33:57.596677 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-5b1c5c2e-d00d-4f09-aa84-d83056182d46 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:33:57.732026 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-5b1c5c2e-d00d-4f09-aa84-d83056182d46 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:33:57.740465 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-5b1c5c2e-d00d-4f09-aa84-d83056182d46 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:33:57.742020 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-5b1c5c2e-d00d-4f09-aa84-d83056182d46 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:33:57.743188 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-5b1c5c2e-d00d-4f09-aa84-d83056182d46 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] 10.222.0.52 "GET /compute/v2.1/servers/fff5c233-ad7f-4022-8582-e8620f32ebd5" status: 200 len: 1732 microversion: 2.20 time: 0.160879 Aug 27 02:33:57.743741 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 490/986] 10.222.0.52 () {60 vars in 1277 bytes} [Sat Aug 27 02:33:57 2022] GET /compute/v2.1/servers/fff5c233-ad7f-4022-8582-e8620f32ebd5 => generated 1732 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 27 02:33:58.765961 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-86b6f2e2-3fa8-4f76-b5e1-046ca4a62101 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Calling method '>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:33:58.766563 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-86b6f2e2-3fa8-4f76-b5e1-046ca4a62101 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] [instance: fff5c233-ad7f-4022-8582-e8620f32ebd5] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:33:58.775024 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-86b6f2e2-3fa8-4f76-b5e1-046ca4a62101 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:33:58.775513 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-86b6f2e2-3fa8-4f76-b5e1-046ca4a62101 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:33:58.775930 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-86b6f2e2-3fa8-4f76-b5e1-046ca4a62101 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:33:58.776637 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-86b6f2e2-3fa8-4f76-b5e1-046ca4a62101 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:33:58.777093 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-86b6f2e2-3fa8-4f76-b5e1-046ca4a62101 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:33:58.777540 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-86b6f2e2-3fa8-4f76-b5e1-046ca4a62101 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:33:58.910566 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-86b6f2e2-3fa8-4f76-b5e1-046ca4a62101 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:33:58.919006 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-86b6f2e2-3fa8-4f76-b5e1-046ca4a62101 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:33:58.920452 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-86b6f2e2-3fa8-4f76-b5e1-046ca4a62101 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:33:58.921645 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-86b6f2e2-3fa8-4f76-b5e1-046ca4a62101 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] 10.222.0.52 "GET /compute/v2.1/servers/fff5c233-ad7f-4022-8582-e8620f32ebd5" status: 200 len: 1732 microversion: 2.20 time: 0.159619 Aug 27 02:33:58.922179 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 497/987] 10.222.0.52 () {60 vars in 1277 bytes} [Sat Aug 27 02:33:58 2022] GET /compute/v2.1/servers/fff5c233-ad7f-4022-8582-e8620f32ebd5 => generated 1732 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 27 02:33:59.944960 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-196cbf49-9e9a-47f5-a2e4-802d2cca2b7b tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:33:59.945609 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-196cbf49-9e9a-47f5-a2e4-802d2cca2b7b tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] [instance: fff5c233-ad7f-4022-8582-e8620f32ebd5] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:33:59.952338 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-196cbf49-9e9a-47f5-a2e4-802d2cca2b7b tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:33:59.952828 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-196cbf49-9e9a-47f5-a2e4-802d2cca2b7b tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:33:59.953318 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-196cbf49-9e9a-47f5-a2e4-802d2cca2b7b tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:33:59.954020 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-196cbf49-9e9a-47f5-a2e4-802d2cca2b7b tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:33:59.954501 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-196cbf49-9e9a-47f5-a2e4-802d2cca2b7b tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:33:59.954925 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-196cbf49-9e9a-47f5-a2e4-802d2cca2b7b tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:34:00.094896 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-196cbf49-9e9a-47f5-a2e4-802d2cca2b7b tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:34:00.103645 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-196cbf49-9e9a-47f5-a2e4-802d2cca2b7b tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:34:00.105089 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-196cbf49-9e9a-47f5-a2e4-802d2cca2b7b tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:34:00.106274 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-196cbf49-9e9a-47f5-a2e4-802d2cca2b7b tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] 10.222.0.52 "GET /compute/v2.1/servers/fff5c233-ad7f-4022-8582-e8620f32ebd5" status: 200 len: 1732 microversion: 2.20 time: 0.165464 Aug 27 02:34:00.106841 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 491/988] 10.222.0.52 () {60 vars in 1277 bytes} [Sat Aug 27 02:33:59 2022] GET /compute/v2.1/servers/fff5c233-ad7f-4022-8582-e8620f32ebd5 => generated 1732 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 27 02:34:01.130310 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-5b17d8b2-c4c9-4ecd-be89-b2a2a97849f4 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Calling method '>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:34:01.131079 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-5b17d8b2-c4c9-4ecd-be89-b2a2a97849f4 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] [instance: fff5c233-ad7f-4022-8582-e8620f32ebd5] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:34:01.138667 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-5b17d8b2-c4c9-4ecd-be89-b2a2a97849f4 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:34:01.139116 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-5b17d8b2-c4c9-4ecd-be89-b2a2a97849f4 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:34:01.139542 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-5b17d8b2-c4c9-4ecd-be89-b2a2a97849f4 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:34:01.140252 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-5b17d8b2-c4c9-4ecd-be89-b2a2a97849f4 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:34:01.140703 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-5b17d8b2-c4c9-4ecd-be89-b2a2a97849f4 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:34:01.141156 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-5b17d8b2-c4c9-4ecd-be89-b2a2a97849f4 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:34:01.286605 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-5b17d8b2-c4c9-4ecd-be89-b2a2a97849f4 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:34:01.295505 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-5b17d8b2-c4c9-4ecd-be89-b2a2a97849f4 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:34:01.297086 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-5b17d8b2-c4c9-4ecd-be89-b2a2a97849f4 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:34:01.298474 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-5b17d8b2-c4c9-4ecd-be89-b2a2a97849f4 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] 10.222.0.52 "GET /compute/v2.1/servers/fff5c233-ad7f-4022-8582-e8620f32ebd5" status: 200 len: 1732 microversion: 2.20 time: 0.172931 Aug 27 02:34:01.299477 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 498/989] 10.222.0.52 () {60 vars in 1277 bytes} [Sat Aug 27 02:34:01 2022] GET /compute/v2.1/servers/fff5c233-ad7f-4022-8582-e8620f32ebd5 => generated 1732 bytes in 174 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 27 02:34:02.324073 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-b96fb878-d672-45fa-89df-43a5fd026741 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:34:02.324686 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-b96fb878-d672-45fa-89df-43a5fd026741 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] [instance: fff5c233-ad7f-4022-8582-e8620f32ebd5] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:34:02.331680 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-b96fb878-d672-45fa-89df-43a5fd026741 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:34:02.332175 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-b96fb878-d672-45fa-89df-43a5fd026741 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:34:02.332618 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-b96fb878-d672-45fa-89df-43a5fd026741 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:34:02.333367 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-b96fb878-d672-45fa-89df-43a5fd026741 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:34:02.333822 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-b96fb878-d672-45fa-89df-43a5fd026741 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:34:02.334469 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-b96fb878-d672-45fa-89df-43a5fd026741 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:34:02.483032 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-b96fb878-d672-45fa-89df-43a5fd026741 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:34:02.492543 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-b96fb878-d672-45fa-89df-43a5fd026741 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:34:02.494008 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-b96fb878-d672-45fa-89df-43a5fd026741 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:34:02.495192 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-b96fb878-d672-45fa-89df-43a5fd026741 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] 10.222.0.52 "GET /compute/v2.1/servers/fff5c233-ad7f-4022-8582-e8620f32ebd5" status: 200 len: 1732 microversion: 2.20 time: 0.175164 Aug 27 02:34:02.495742 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 492/990] 10.222.0.52 () {60 vars in 1277 bytes} [Sat Aug 27 02:34:02 2022] GET /compute/v2.1/servers/fff5c233-ad7f-4022-8582-e8620f32ebd5 => generated 1732 bytes in 176 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 27 02:34:03.532817 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-16514e3c-3a91-4702-8a3a-e4ae3df9f4f3 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Calling method '>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:34:03.533464 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-16514e3c-3a91-4702-8a3a-e4ae3df9f4f3 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] [instance: fff5c233-ad7f-4022-8582-e8620f32ebd5] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:34:03.540324 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-16514e3c-3a91-4702-8a3a-e4ae3df9f4f3 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:34:03.540801 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-16514e3c-3a91-4702-8a3a-e4ae3df9f4f3 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:34:03.541376 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-16514e3c-3a91-4702-8a3a-e4ae3df9f4f3 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:34:03.542122 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-16514e3c-3a91-4702-8a3a-e4ae3df9f4f3 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:34:03.542610 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-16514e3c-3a91-4702-8a3a-e4ae3df9f4f3 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:34:03.543032 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-16514e3c-3a91-4702-8a3a-e4ae3df9f4f3 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:34:03.680578 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-16514e3c-3a91-4702-8a3a-e4ae3df9f4f3 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:34:03.688816 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-16514e3c-3a91-4702-8a3a-e4ae3df9f4f3 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:34:03.690452 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-16514e3c-3a91-4702-8a3a-e4ae3df9f4f3 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:34:03.691646 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-16514e3c-3a91-4702-8a3a-e4ae3df9f4f3 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] 10.222.0.52 "GET /compute/v2.1/servers/fff5c233-ad7f-4022-8582-e8620f32ebd5" status: 200 len: 1732 microversion: 2.20 time: 0.162756 Aug 27 02:34:03.692204 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 499/991] 10.222.0.52 () {60 vars in 1277 bytes} [Sat Aug 27 02:34:03 2022] GET /compute/v2.1/servers/fff5c233-ad7f-4022-8582-e8620f32ebd5 => generated 1732 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 27 02:34:04.563831 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [req-ece4e678-c05c-4db2-bf12-bd728e7eb3cb req-56d9647c-31f8-4ff4-8b37-03f7a53e56a0 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "fff5c233-ad7f-4022-8582-e8620f32ebd5", "name": "network-vif-plugged", "status": "completed", "tag": "67fc5822-1682-4324-ae1c-b98d8d2d09e8"}]} {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 27 02:34:04.574189 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [req-ece4e678-c05c-4db2-bf12-bd728e7eb3cb req-56d9647c-31f8-4ff4-8b37-03f7a53e56a0 service nova] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:34:04.574672 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [req-ece4e678-c05c-4db2-bf12-bd728e7eb3cb req-56d9647c-31f8-4ff4-8b37-03f7a53e56a0 service nova] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:34:04.575081 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [req-ece4e678-c05c-4db2-bf12-bd728e7eb3cb req-56d9647c-31f8-4ff4-8b37-03f7a53e56a0 service nova] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:34:04.586301 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.compute.server_external_events [req-ece4e678-c05c-4db2-bf12-bd728e7eb3cb req-56d9647c-31f8-4ff4-8b37-03f7a53e56a0 service nova] Creating event network-vif-plugged:67fc5822-1682-4324-ae1c-b98d8d2d09e8 for instance fff5c233-ad7f-4022-8582-e8620f32ebd5 on hv-cinder-84630 Aug 27 02:34:04.591729 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [req-ece4e678-c05c-4db2-bf12-bd728e7eb3cb req-56d9647c-31f8-4ff4-8b37-03f7a53e56a0 service nova] 10.222.0.52 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.163359 Aug 27 02:34:04.592295 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 493/992] 10.222.0.52 () {64 vars in 1317 bytes} [Sat Aug 27 02:34:04 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 27 02:34:04.715682 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-359c8229-8a6b-4d8a-a2d2-8522515dbc32 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Calling method '>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:34:04.716268 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-359c8229-8a6b-4d8a-a2d2-8522515dbc32 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] [instance: fff5c233-ad7f-4022-8582-e8620f32ebd5] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:34:04.723322 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-359c8229-8a6b-4d8a-a2d2-8522515dbc32 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:34:04.724164 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-359c8229-8a6b-4d8a-a2d2-8522515dbc32 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:34:04.724864 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-359c8229-8a6b-4d8a-a2d2-8522515dbc32 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:34:04.725752 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-359c8229-8a6b-4d8a-a2d2-8522515dbc32 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:34:04.726218 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-359c8229-8a6b-4d8a-a2d2-8522515dbc32 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:34:04.726636 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-359c8229-8a6b-4d8a-a2d2-8522515dbc32 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:34:04.905963 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-359c8229-8a6b-4d8a-a2d2-8522515dbc32 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:34:04.914592 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-359c8229-8a6b-4d8a-a2d2-8522515dbc32 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:34:04.916135 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-359c8229-8a6b-4d8a-a2d2-8522515dbc32 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:34:04.917333 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-359c8229-8a6b-4d8a-a2d2-8522515dbc32 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] 10.222.0.52 "GET /compute/v2.1/servers/fff5c233-ad7f-4022-8582-e8620f32ebd5" status: 200 len: 1732 microversion: 2.20 time: 0.205607 Aug 27 02:34:04.917922 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 500/993] 10.222.0.52 () {60 vars in 1277 bytes} [Sat Aug 27 02:34:04 2022] GET /compute/v2.1/servers/fff5c233-ad7f-4022-8582-e8620f32ebd5 => generated 1732 bytes in 207 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Aug 27 02:34:05.941530 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-9b38b1fc-0fb5-403e-867e-bb7ee2183050 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:34:05.942250 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-9b38b1fc-0fb5-403e-867e-bb7ee2183050 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] [instance: fff5c233-ad7f-4022-8582-e8620f32ebd5] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:34:05.950715 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-9b38b1fc-0fb5-403e-867e-bb7ee2183050 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:34:05.951236 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-9b38b1fc-0fb5-403e-867e-bb7ee2183050 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:34:05.951675 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-9b38b1fc-0fb5-403e-867e-bb7ee2183050 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:34:05.952373 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-9b38b1fc-0fb5-403e-867e-bb7ee2183050 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:34:05.952821 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-9b38b1fc-0fb5-403e-867e-bb7ee2183050 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:34:05.953313 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-9b38b1fc-0fb5-403e-867e-bb7ee2183050 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:34:06.090772 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-9b38b1fc-0fb5-403e-867e-bb7ee2183050 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:34:06.100002 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-9b38b1fc-0fb5-403e-867e-bb7ee2183050 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:34:06.101913 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-9b38b1fc-0fb5-403e-867e-bb7ee2183050 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:34:06.103262 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-9b38b1fc-0fb5-403e-867e-bb7ee2183050 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] 10.222.0.52 "GET /compute/v2.1/servers/fff5c233-ad7f-4022-8582-e8620f32ebd5" status: 200 len: 1732 microversion: 2.20 time: 0.166679 Aug 27 02:34:06.103808 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 494/994] 10.222.0.52 () {60 vars in 1277 bytes} [Sat Aug 27 02:34:05 2022] GET /compute/v2.1/servers/fff5c233-ad7f-4022-8582-e8620f32ebd5 => generated 1732 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 27 02:34:07.126301 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-48793aaf-2e8b-41e0-9cfc-74694b3ab5bb tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Calling method '>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:34:07.126912 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-48793aaf-2e8b-41e0-9cfc-74694b3ab5bb tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] [instance: fff5c233-ad7f-4022-8582-e8620f32ebd5] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:34:07.133732 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-48793aaf-2e8b-41e0-9cfc-74694b3ab5bb tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:34:07.134190 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-48793aaf-2e8b-41e0-9cfc-74694b3ab5bb tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:34:07.134621 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-48793aaf-2e8b-41e0-9cfc-74694b3ab5bb tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:34:07.135301 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-48793aaf-2e8b-41e0-9cfc-74694b3ab5bb tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:34:07.135740 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-48793aaf-2e8b-41e0-9cfc-74694b3ab5bb tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:34:07.136154 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-48793aaf-2e8b-41e0-9cfc-74694b3ab5bb tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:34:07.281023 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-48793aaf-2e8b-41e0-9cfc-74694b3ab5bb tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:34:07.289401 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-48793aaf-2e8b-41e0-9cfc-74694b3ab5bb tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:34:07.290823 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-48793aaf-2e8b-41e0-9cfc-74694b3ab5bb tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:34:07.291966 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-48793aaf-2e8b-41e0-9cfc-74694b3ab5bb tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] 10.222.0.52 "GET /compute/v2.1/servers/fff5c233-ad7f-4022-8582-e8620f32ebd5" status: 200 len: 1719 microversion: 2.20 time: 0.169621 Aug 27 02:34:07.292532 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 501/995] 10.222.0.52 () {60 vars in 1277 bytes} [Sat Aug 27 02:34:07 2022] GET /compute/v2.1/servers/fff5c233-ad7f-4022-8582-e8620f32ebd5 => generated 1719 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 27 02:34:14.204229 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-84b54a95-7166-45c9-b315-379786ee4425 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:34:14.204958 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-84b54a95-7166-45c9-b315-379786ee4425 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] [instance: 77068029-afdb-4560-8d56-76c56b85ef3a] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:34:14.213048 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-84b54a95-7166-45c9-b315-379786ee4425 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:34:14.213636 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-84b54a95-7166-45c9-b315-379786ee4425 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:34:14.214132 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-84b54a95-7166-45c9-b315-379786ee4425 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:34:14.214911 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-84b54a95-7166-45c9-b315-379786ee4425 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:34:14.215409 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-84b54a95-7166-45c9-b315-379786ee4425 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:34:14.215877 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-84b54a95-7166-45c9-b315-379786ee4425 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:34:14.242310 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-84b54a95-7166-45c9-b315-379786ee4425 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] 10.222.0.52 "GET /compute/v2.1/servers/77068029-afdb-4560-8d56-76c56b85ef3a/ips" status: 200 len: 149 microversion: 2.1 time: 0.043783 Aug 27 02:34:14.242871 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 495/996] 10.222.0.52 () {58 vars in 1248 bytes} [Sat Aug 27 02:34:14 2022] GET /compute/v2.1/servers/77068029-afdb-4560-8d56-76c56b85ef3a/ips => generated 149 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 27 02:34:15.623618 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-d11148e9-5648-487b-9589-9b6194a37c4a tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "21bc3b4c-62b8-47f7-b2b6-217f050311c6"}} {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 27 02:34:15.626341 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-d11148e9-5648-487b-9589-9b6194a37c4a tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] [instance: 77068029-afdb-4560-8d56-76c56b85ef3a] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:34:15.633905 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-d11148e9-5648-487b-9589-9b6194a37c4a tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:34:15.634390 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-d11148e9-5648-487b-9589-9b6194a37c4a tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:34:15.634818 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-d11148e9-5648-487b-9589-9b6194a37c4a tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:34:15.635523 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-d11148e9-5648-487b-9589-9b6194a37c4a tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:34:15.635962 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-d11148e9-5648-487b-9589-9b6194a37c4a tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:34:15.636378 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-d11148e9-5648-487b-9589-9b6194a37c4a tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:34:15.967047 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-d11148e9-5648-487b-9589-9b6194a37c4a tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] 10.222.0.52 "POST /compute/v2.1/servers/77068029-afdb-4560-8d56-76c56b85ef3a/os-volume_attachments" status: 200 len: 194 microversion: 2.1 time: 0.348957 Aug 27 02:34:15.967677 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 502/997] 10.222.0.52 () {60 vars in 1323 bytes} [Sat Aug 27 02:34:15 2022] POST /compute/v2.1/servers/77068029-afdb-4560-8d56-76c56b85ef3a/os-volume_attachments => generated 194 bytes in 351 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Aug 27 02:34:23.702044 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-fe4b7f77-ff3d-4bc0-ba78-689e4b1e7f9b tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:34:23.702665 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-fe4b7f77-ff3d-4bc0-ba78-689e4b1e7f9b tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] [instance: 77068029-afdb-4560-8d56-76c56b85ef3a] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:34:23.707711 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG dbcounter [-] [120863] Writing DB stats nova_api:SELECT=16 {{(pid=120863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 27 02:34:23.714740 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-fe4b7f77-ff3d-4bc0-ba78-689e4b1e7f9b tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:34:23.715762 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-fe4b7f77-ff3d-4bc0-ba78-689e4b1e7f9b tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:34:23.715762 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-fe4b7f77-ff3d-4bc0-ba78-689e4b1e7f9b tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:34:23.716469 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-fe4b7f77-ff3d-4bc0-ba78-689e4b1e7f9b tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:34:23.716924 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-fe4b7f77-ff3d-4bc0-ba78-689e4b1e7f9b tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:34:23.717966 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-fe4b7f77-ff3d-4bc0-ba78-689e4b1e7f9b tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:34:23.720474 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG dbcounter [-] [120863] Writing DB stats nova_cell1:SELECT=42 {{(pid=120863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 27 02:34:23.819713 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.objects.instance [None req-fe4b7f77-ff3d-4bc0-ba78-689e4b1e7f9b tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lazy-loading 'services' on Instance uuid 77068029-afdb-4560-8d56-76c56b85ef3a {{(pid=120863) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Aug 27 02:34:23.906413 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-fe4b7f77-ff3d-4bc0-ba78-689e4b1e7f9b tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] 10.222.0.52 "DELETE /compute/v2.1/servers/77068029-afdb-4560-8d56-76c56b85ef3a/os-volume_attachments/21bc3b4c-62b8-47f7-b2b6-217f050311c6" status: 202 len: 0 microversion: 2.1 time: 0.209893 Aug 27 02:34:23.907012 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 496/998] 10.222.0.52 () {58 vars in 1416 bytes} [Sat Aug 27 02:34:23 2022] DELETE /compute/v2.1/servers/77068029-afdb-4560-8d56-76c56b85ef3a/os-volume_attachments/21bc3b4c-62b8-47f7-b2b6-217f050311c6 => generated 0 bytes in 212 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Aug 27 02:34:23.917967 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-081c8d1d-8a05-4d75-9770-6263ab8068d4 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Calling method '>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:34:23.920593 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-081c8d1d-8a05-4d75-9770-6263ab8068d4 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] [instance: 77068029-afdb-4560-8d56-76c56b85ef3a] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:34:23.927589 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-081c8d1d-8a05-4d75-9770-6263ab8068d4 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:34:23.928149 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-081c8d1d-8a05-4d75-9770-6263ab8068d4 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:34:23.928646 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-081c8d1d-8a05-4d75-9770-6263ab8068d4 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:34:23.929459 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-081c8d1d-8a05-4d75-9770-6263ab8068d4 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:34:23.929897 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-081c8d1d-8a05-4d75-9770-6263ab8068d4 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:34:23.931397 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-081c8d1d-8a05-4d75-9770-6263ab8068d4 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:34:23.973086 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-081c8d1d-8a05-4d75-9770-6263ab8068d4 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] 10.222.0.52 "GET /compute/v2.1/servers/77068029-afdb-4560-8d56-76c56b85ef3a/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.060109 Aug 27 02:34:23.973838 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 503/999] 10.222.0.52 () {58 vars in 1302 bytes} [Sat Aug 27 02:34:23 2022] GET /compute/v2.1/servers/77068029-afdb-4560-8d56-76c56b85ef3a/os-volume_attachments => generated 197 bytes in 62 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 27 02:34:24.991779 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-ac645ba6-44d9-4317-8442-150e0bed80b2 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:34:24.994203 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-ac645ba6-44d9-4317-8442-150e0bed80b2 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] [instance: 77068029-afdb-4560-8d56-76c56b85ef3a] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:34:25.004252 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-ac645ba6-44d9-4317-8442-150e0bed80b2 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:34:25.005045 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-ac645ba6-44d9-4317-8442-150e0bed80b2 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:34:25.005806 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-ac645ba6-44d9-4317-8442-150e0bed80b2 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:34:25.007063 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-ac645ba6-44d9-4317-8442-150e0bed80b2 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:34:25.007807 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-ac645ba6-44d9-4317-8442-150e0bed80b2 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:34:25.008498 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-ac645ba6-44d9-4317-8442-150e0bed80b2 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:34:25.041866 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-ac645ba6-44d9-4317-8442-150e0bed80b2 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] 10.222.0.52 "GET /compute/v2.1/servers/77068029-afdb-4560-8d56-76c56b85ef3a/os-volume_attachments" status: 200 len: 25 microversion: 2.1 time: 0.058408 Aug 27 02:34:25.042450 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 497/1000] 10.222.0.52 () {58 vars in 1302 bytes} [Sat Aug 27 02:34:24 2022] GET /compute/v2.1/servers/77068029-afdb-4560-8d56-76c56b85ef3a/os-volume_attachments => generated 25 bytes in 60 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Aug 27 02:34:25.115802 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-520d7a9a-5705-4fe4-869e-118cc7b50d60 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Calling method '>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:34:25.116446 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-520d7a9a-5705-4fe4-869e-118cc7b50d60 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] [instance: 77068029-afdb-4560-8d56-76c56b85ef3a] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:34:25.123864 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-520d7a9a-5705-4fe4-869e-118cc7b50d60 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:34:25.124506 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-520d7a9a-5705-4fe4-869e-118cc7b50d60 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:34:25.124966 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-520d7a9a-5705-4fe4-869e-118cc7b50d60 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:34:25.125787 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-520d7a9a-5705-4fe4-869e-118cc7b50d60 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:34:25.126242 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-520d7a9a-5705-4fe4-869e-118cc7b50d60 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:34:25.126700 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-520d7a9a-5705-4fe4-869e-118cc7b50d60 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:34:25.129705 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG dbcounter [-] [120864] Writing DB stats nova_cell1:SELECT=61,nova_cell1:UPDATE=2,nova_cell1:INSERT=1 {{(pid=120864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 27 02:34:25.158103 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-520d7a9a-5705-4fe4-869e-118cc7b50d60 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] [instance: 77068029-afdb-4560-8d56-76c56b85ef3a] Going to try to terminate instance {{(pid=120864) delete /opt/stack/nova/nova/compute/api.py:2719}} Aug 27 02:34:25.252855 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-520d7a9a-5705-4fe4-869e-118cc7b50d60 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] 10.222.0.52 "DELETE /compute/v2.1/servers/77068029-afdb-4560-8d56-76c56b85ef3a" status: 204 len: 0 microversion: 2.1 time: 0.140817 Aug 27 02:34:25.253431 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 504/1001] 10.222.0.52 () {58 vars in 1239 bytes} [Sat Aug 27 02:34:25 2022] DELETE /compute/v2.1/servers/77068029-afdb-4560-8d56-76c56b85ef3a => generated 0 bytes in 142 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Aug 27 02:34:25.265867 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-48ae9deb-3280-453a-8194-640b7fc59a81 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:34:25.266489 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-48ae9deb-3280-453a-8194-640b7fc59a81 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] [instance: 77068029-afdb-4560-8d56-76c56b85ef3a] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:34:25.272847 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-48ae9deb-3280-453a-8194-640b7fc59a81 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:34:25.273375 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-48ae9deb-3280-453a-8194-640b7fc59a81 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:34:25.273821 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-48ae9deb-3280-453a-8194-640b7fc59a81 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:34:25.274649 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-48ae9deb-3280-453a-8194-640b7fc59a81 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:34:25.275103 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-48ae9deb-3280-453a-8194-640b7fc59a81 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:34:25.275529 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-48ae9deb-3280-453a-8194-640b7fc59a81 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:34:25.436518 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-48ae9deb-3280-453a-8194-640b7fc59a81 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '26a86351583d49d090b3db09f127d49f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5d5ec8450c0941d18d63558baab54737', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:34:25.446701 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-48ae9deb-3280-453a-8194-640b7fc59a81 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] 10.222.0.52 "GET /compute/v2.1/servers/77068029-afdb-4560-8d56-76c56b85ef3a" status: 200 len: 1680 microversion: 2.1 time: 0.186198 Aug 27 02:34:25.447475 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 498/1002] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:34:25 2022] GET /compute/v2.1/servers/77068029-afdb-4560-8d56-76c56b85ef3a => generated 1680 bytes in 189 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:34:26.471218 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-cdbb516b-11d8-4045-a0e1-3345ecc9979f tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Calling method '>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:34:26.471962 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-cdbb516b-11d8-4045-a0e1-3345ecc9979f tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] [instance: 77068029-afdb-4560-8d56-76c56b85ef3a] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:34:26.474721 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG dbcounter [-] [120864] Writing DB stats nova_api:SELECT=23,nova_api:UPDATE=1 {{(pid=120864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 27 02:34:26.482421 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-cdbb516b-11d8-4045-a0e1-3345ecc9979f tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:34:26.483062 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-cdbb516b-11d8-4045-a0e1-3345ecc9979f tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:34:26.483549 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-cdbb516b-11d8-4045-a0e1-3345ecc9979f tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:34:26.484252 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-cdbb516b-11d8-4045-a0e1-3345ecc9979f tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:34:26.484696 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-cdbb516b-11d8-4045-a0e1-3345ecc9979f tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:34:26.485109 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-cdbb516b-11d8-4045-a0e1-3345ecc9979f tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:34:26.646752 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-cdbb516b-11d8-4045-a0e1-3345ecc9979f tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '26a86351583d49d090b3db09f127d49f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5d5ec8450c0941d18d63558baab54737', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:34:26.658505 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-cdbb516b-11d8-4045-a0e1-3345ecc9979f tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] 10.222.0.52 "GET /compute/v2.1/servers/77068029-afdb-4560-8d56-76c56b85ef3a" status: 200 len: 1680 microversion: 2.1 time: 0.191820 Aug 27 02:34:26.659114 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 505/1003] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:34:26 2022] GET /compute/v2.1/servers/77068029-afdb-4560-8d56-76c56b85ef3a => generated 1680 bytes in 193 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:34:27.681972 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-50ec1cfc-51a3-4451-a147-23b19a24ede1 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:34:27.682563 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-50ec1cfc-51a3-4451-a147-23b19a24ede1 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] [instance: 77068029-afdb-4560-8d56-76c56b85ef3a] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:34:27.690046 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-50ec1cfc-51a3-4451-a147-23b19a24ede1 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:34:27.690665 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-50ec1cfc-51a3-4451-a147-23b19a24ede1 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:34:27.691303 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-50ec1cfc-51a3-4451-a147-23b19a24ede1 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:34:27.692249 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-50ec1cfc-51a3-4451-a147-23b19a24ede1 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:34:27.692769 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-50ec1cfc-51a3-4451-a147-23b19a24ede1 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:34:27.693265 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-50ec1cfc-51a3-4451-a147-23b19a24ede1 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:34:27.926761 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-50ec1cfc-51a3-4451-a147-23b19a24ede1 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '26a86351583d49d090b3db09f127d49f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5d5ec8450c0941d18d63558baab54737', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:34:27.941163 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-50ec1cfc-51a3-4451-a147-23b19a24ede1 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] 10.222.0.52 "GET /compute/v2.1/servers/77068029-afdb-4560-8d56-76c56b85ef3a" status: 200 len: 1680 microversion: 2.1 time: 0.262797 Aug 27 02:34:27.941883 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 499/1004] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:34:27 2022] GET /compute/v2.1/servers/77068029-afdb-4560-8d56-76c56b85ef3a => generated 1680 bytes in 265 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:34:28.587172 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [req-79b3a522-fdff-4aca-98d0-cdbb961604f7 req-9eac5e62-dfd8-4728-99a1-b48cec7389ba service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "77068029-afdb-4560-8d56-76c56b85ef3a", "name": "network-vif-deleted", "tag": "46a7cd9a-679a-4cbc-95b2-41424b3783c2"}]} {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 27 02:34:28.604280 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [req-79b3a522-fdff-4aca-98d0-cdbb961604f7 req-9eac5e62-dfd8-4728-99a1-b48cec7389ba service nova] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:34:28.604845 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [req-79b3a522-fdff-4aca-98d0-cdbb961604f7 req-9eac5e62-dfd8-4728-99a1-b48cec7389ba service nova] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:34:28.605399 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [req-79b3a522-fdff-4aca-98d0-cdbb961604f7 req-9eac5e62-dfd8-4728-99a1-b48cec7389ba service nova] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:34:28.620942 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.compute.server_external_events [req-79b3a522-fdff-4aca-98d0-cdbb961604f7 req-9eac5e62-dfd8-4728-99a1-b48cec7389ba service nova] Creating event network-vif-deleted:46a7cd9a-679a-4cbc-95b2-41424b3783c2 for instance 77068029-afdb-4560-8d56-76c56b85ef3a on hv-cinder-84630 Aug 27 02:34:28.629529 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [req-79b3a522-fdff-4aca-98d0-cdbb961604f7 req-9eac5e62-dfd8-4728-99a1-b48cec7389ba service nova] 10.222.0.52 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.046769 Aug 27 02:34:28.632122 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 506/1005] 10.222.0.52 () {64 vars in 1317 bytes} [Sat Aug 27 02:34:28 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 52 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 27 02:34:28.966267 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-cd6b0bf1-2c8f-4a69-b740-88ba73dd5fca tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:34:28.967159 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-cd6b0bf1-2c8f-4a69-b740-88ba73dd5fca tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] [instance: 77068029-afdb-4560-8d56-76c56b85ef3a] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:34:28.973573 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-cd6b0bf1-2c8f-4a69-b740-88ba73dd5fca tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:34:28.974121 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-cd6b0bf1-2c8f-4a69-b740-88ba73dd5fca tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:34:28.974613 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-cd6b0bf1-2c8f-4a69-b740-88ba73dd5fca tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:34:28.975708 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-cd6b0bf1-2c8f-4a69-b740-88ba73dd5fca tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:34:28.976152 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-cd6b0bf1-2c8f-4a69-b740-88ba73dd5fca tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:34:28.976585 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-cd6b0bf1-2c8f-4a69-b740-88ba73dd5fca tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:34:29.016179 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-cd6b0bf1-2c8f-4a69-b740-88ba73dd5fca tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:34:29.016439 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-cd6b0bf1-2c8f-4a69-b740-88ba73dd5fca tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:34:29.016912 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-cd6b0bf1-2c8f-4a69-b740-88ba73dd5fca tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:34:29.017407 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.objects.instance [None req-cd6b0bf1-2c8f-4a69-b740-88ba73dd5fca tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lazy-loading 'fault' on Instance uuid 77068029-afdb-4560-8d56-76c56b85ef3a {{(pid=120863) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Aug 27 02:34:29.064017 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-cd6b0bf1-2c8f-4a69-b740-88ba73dd5fca tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '26a86351583d49d090b3db09f127d49f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5d5ec8450c0941d18d63558baab54737', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:34:29.073164 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-cd6b0bf1-2c8f-4a69-b740-88ba73dd5fca tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] 10.222.0.52 "GET /compute/v2.1/servers/77068029-afdb-4560-8d56-76c56b85ef3a" status: 200 len: 1331 microversion: 2.1 time: 0.111256 Aug 27 02:34:29.073807 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 500/1006] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:34:28 2022] GET /compute/v2.1/servers/77068029-afdb-4560-8d56-76c56b85ef3a => generated 1331 bytes in 113 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:34:30.096974 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-39aaf81f-4f4c-48bf-8968-231c1e9c7074 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Calling method '>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:34:30.097739 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-39aaf81f-4f4c-48bf-8968-231c1e9c7074 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] [instance: 77068029-afdb-4560-8d56-76c56b85ef3a] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:34:30.104655 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-39aaf81f-4f4c-48bf-8968-231c1e9c7074 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:34:30.105158 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-39aaf81f-4f4c-48bf-8968-231c1e9c7074 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:34:30.105595 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-39aaf81f-4f4c-48bf-8968-231c1e9c7074 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:34:30.106320 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-39aaf81f-4f4c-48bf-8968-231c1e9c7074 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:34:30.106996 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-39aaf81f-4f4c-48bf-8968-231c1e9c7074 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:34:30.107416 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-39aaf81f-4f4c-48bf-8968-231c1e9c7074 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:34:30.123764 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.wsgi [None req-39aaf81f-4f4c-48bf-8968-231c1e9c7074 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] HTTP exception thrown: Instance 77068029-afdb-4560-8d56-76c56b85ef3a could not be found. Aug 27 02:34:30.128294 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-39aaf81f-4f4c-48bf-8968-231c1e9c7074 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Request method failure captured: Aug 27 02:34:30.128294 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: request: GET /compute/v2.1/servers/77068029-afdb-4560-8d56-76c56b85ef3a HTTP/1.1 Aug 27 02:34:30.128294 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: Accept: application/json Aug 27 02:34:30.128294 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: Accept-Encoding: identity Aug 27 02:34:30.128294 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: Connection: close Aug 27 02:34:30.128294 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: Content-Length: 0 Aug 27 02:34:30.128294 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: Content-Type: application/json Aug 27 02:34:30.128294 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: Host: 10.222.0.52 Aug 27 02:34:30.128294 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: Openstack-System-Scope: None Aug 27 02:34:30.128294 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: User-Agent: python-urllib3/1.26.11 Aug 27 02:34:30.128294 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-Auth-Token: gAAAAABjCX_pkOb4-Fffq6Hbe0cfjlj1ARL1JmORSzrYWucWmWgZ67LEorDD3IWpPkj3UP534US8GW_nja0iAKwcdxSx3e5teQzZMdrCvOvA5aSGJO8iodOjhfOIsl8L0d8om8kj8_3q-8uTHxKupSTnIQy_J-f1kC4e22VNejIIiExDsGitats Aug 27 02:34:30.128294 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-Domain-Id: None Aug 27 02:34:30.128294 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-Domain-Name: None Aug 27 02:34:30.128294 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-Identity-Status: Confirmed Aug 27 02:34:30.128294 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-Is-Admin-Project: True Aug 27 02:34:30.128294 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-Project-Domain-Id: default Aug 27 02:34:30.128294 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-Project-Domain-Name: Default Aug 27 02:34:30.128294 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-Project-Id: 5d5ec8450c0941d18d63558baab54737 Aug 27 02:34:30.128294 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-Project-Name: tempest-AttachVolumeNegativeTest-1988669364 Aug 27 02:34:30.128294 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-Role: reader,member Aug 27 02:34:30.128294 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-Roles: reader,member Aug 27 02:34:30.130017 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-Service-Catalog: [{"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.52:8080", "publicURL": "http://10.222.0.52:8080/v1/AUTH_5d5ec8450c0941d18d63558baab54737"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52:9696/networking"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/compute/v2/5d5ec8450c0941d18d63558baab54737"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/image"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/volume/v3/5d5ec8450c0941d18d63558baab54737"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/identity"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/volume/v3/5d5ec8450c0941d18d63558baab54737"}]}] Aug 27 02:34:30.130017 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-Tenant: tempest-AttachVolumeNegativeTest-1988669364 Aug 27 02:34:30.130017 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-Tenant-Id: 5d5ec8450c0941d18d63558baab54737 Aug 27 02:34:30.130017 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-1988669364 Aug 27 02:34:30.130017 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-User: tempest-AttachVolumeNegativeTest-1988669364-project Aug 27 02:34:30.130017 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-User-Domain-Id: default Aug 27 02:34:30.130017 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-User-Domain-Name: Default Aug 27 02:34:30.130017 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-User-Id: 26a86351583d49d090b3db09f127d49f Aug 27 02:34:30.130017 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-User-Name: tempest-AttachVolumeNegativeTest-1988669364-project Aug 27 02:34:30.130017 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: method: > Aug 27 02:34:30.130017 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: exception: Instance 77068029-afdb-4560-8d56-76c56b85ef3a could not be found. Aug 27 02:34:30.130017 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: action_args: {'id': '77068029-afdb-4560-8d56-76c56b85ef3a'} Aug 27 02:34:30.130916 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Aug 27 02:34:30.130916 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 27 02:34:30.130916 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Aug 27 02:34:30.130916 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Aug 27 02:34:30.130916 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Aug 27 02:34:30.130916 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Aug 27 02:34:30.130916 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Aug 27 02:34:30.130916 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Aug 27 02:34:30.130916 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Aug 27 02:34:30.130916 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Aug 27 02:34:30.130916 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 77068029-afdb-4560-8d56-76c56b85ef3a could not be found. Aug 27 02:34:30.130916 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi Aug 27 02:34:30.130916 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 27 02:34:30.130916 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi Aug 27 02:34:30.130916 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 27 02:34:30.130916 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 27 02:34:30.130916 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 27 02:34:30.130916 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Aug 27 02:34:30.130916 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Aug 27 02:34:30.130916 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Aug 27 02:34:30.132491 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Aug 27 02:34:30.132491 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Aug 27 02:34:30.132491 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi instance = self._get_server( Aug 27 02:34:30.132491 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Aug 27 02:34:30.132491 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Aug 27 02:34:30.132491 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Aug 27 02:34:30.132491 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Aug 27 02:34:30.132491 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 77068029-afdb-4560-8d56-76c56b85ef3a could not be found. Aug 27 02:34:30.132491 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi Aug 27 02:34:30.132491 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 27 02:34:30.132491 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi Aug 27 02:34:30.132491 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 27 02:34:30.132491 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 27 02:34:30.132491 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 27 02:34:30.132491 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Aug 27 02:34:30.132491 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Aug 27 02:34:30.132491 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 77068029-afdb-4560-8d56-76c56b85ef3a could not be found. Aug 27 02:34:30.132491 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi Aug 27 02:34:30.133966 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-39aaf81f-4f4c-48bf-8968-231c1e9c7074 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Returning 404 to user: Instance 77068029-afdb-4560-8d56-76c56b85ef3a could not be found. {{(pid=120864) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Aug 27 02:34:30.133966 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-39aaf81f-4f4c-48bf-8968-231c1e9c7074 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] 10.222.0.52 "GET /compute/v2.1/servers/77068029-afdb-4560-8d56-76c56b85ef3a" status: 404 len: 111 microversion: 2.1 time: 0.037668 Aug 27 02:34:30.133966 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 507/1007] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:34:30 2022] GET /compute/v2.1/servers/77068029-afdb-4560-8d56-76c56b85ef3a => generated 111 bytes in 39 msecs (HTTP/1.1 404) 9 headers in 379 bytes (2 switches on core 0) Aug 27 02:34:30.139623 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-46de7b5e-d619-46f1-863a-2224eb689115 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Calling method '.version_select at 0x7f6b1b1ba670>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:34:30.160206 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-46de7b5e-d619-46f1-863a-2224eb689115 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] 10.222.0.52 "DELETE /compute/v2.1/os-keypairs/tempest-keypair-835171988" status: 202 len: 0 microversion: 2.1 time: 0.024882 Aug 27 02:34:30.160800 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 501/1008] 10.222.0.52 () {58 vars in 1218 bytes} [Sat Aug 27 02:34:30 2022] DELETE /compute/v2.1/os-keypairs/tempest-keypair-835171988 => generated 0 bytes in 26 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Aug 27 02:34:32.023743 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-9bb2e782-dfcc-4c2f-afa8-05b4d948b8db tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Calling method '>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:34:32.024334 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-9bb2e782-dfcc-4c2f-afa8-05b4d948b8db tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] [instance: 77068029-afdb-4560-8d56-76c56b85ef3a] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:34:32.031997 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-9bb2e782-dfcc-4c2f-afa8-05b4d948b8db tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:34:32.032484 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-9bb2e782-dfcc-4c2f-afa8-05b4d948b8db tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:34:32.032917 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-9bb2e782-dfcc-4c2f-afa8-05b4d948b8db tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:34:32.033655 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-9bb2e782-dfcc-4c2f-afa8-05b4d948b8db tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:34:32.034104 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-9bb2e782-dfcc-4c2f-afa8-05b4d948b8db tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:34:32.034528 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-9bb2e782-dfcc-4c2f-afa8-05b4d948b8db tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:34:32.048810 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.wsgi [None req-9bb2e782-dfcc-4c2f-afa8-05b4d948b8db tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] HTTP exception thrown: Instance 77068029-afdb-4560-8d56-76c56b85ef3a could not be found. Aug 27 02:34:32.050433 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-9bb2e782-dfcc-4c2f-afa8-05b4d948b8db tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Request method failure captured: Aug 27 02:34:32.050433 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: request: DELETE /compute/v2.1/servers/77068029-afdb-4560-8d56-76c56b85ef3a HTTP/1.1 Aug 27 02:34:32.050433 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: Accept: application/json Aug 27 02:34:32.050433 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: Accept-Encoding: identity Aug 27 02:34:32.050433 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: Connection: close Aug 27 02:34:32.050433 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: Content-Length: 0 Aug 27 02:34:32.050433 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: Content-Type: application/json Aug 27 02:34:32.050433 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: Host: 10.222.0.52 Aug 27 02:34:32.050433 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: Openstack-System-Scope: None Aug 27 02:34:32.050433 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: User-Agent: python-urllib3/1.26.11 Aug 27 02:34:32.050433 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-Auth-Token: gAAAAABjCX_pkOb4-Fffq6Hbe0cfjlj1ARL1JmORSzrYWucWmWgZ67LEorDD3IWpPkj3UP534US8GW_nja0iAKwcdxSx3e5teQzZMdrCvOvA5aSGJO8iodOjhfOIsl8L0d8om8kj8_3q-8uTHxKupSTnIQy_J-f1kC4e22VNejIIiExDsGitats Aug 27 02:34:32.050433 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-Domain-Id: None Aug 27 02:34:32.050433 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-Domain-Name: None Aug 27 02:34:32.050433 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-Identity-Status: Confirmed Aug 27 02:34:32.050433 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-Is-Admin-Project: True Aug 27 02:34:32.050433 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-Project-Domain-Id: default Aug 27 02:34:32.050433 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-Project-Domain-Name: Default Aug 27 02:34:32.050433 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-Project-Id: 5d5ec8450c0941d18d63558baab54737 Aug 27 02:34:32.050433 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-Project-Name: tempest-AttachVolumeNegativeTest-1988669364 Aug 27 02:34:32.050433 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-Role: reader,member Aug 27 02:34:32.050433 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-Roles: reader,member Aug 27 02:34:32.052028 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-Service-Catalog: [{"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.52:8080", "publicURL": "http://10.222.0.52:8080/v1/AUTH_5d5ec8450c0941d18d63558baab54737"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52:9696/networking"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/compute/v2/5d5ec8450c0941d18d63558baab54737"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/image"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/volume/v3/5d5ec8450c0941d18d63558baab54737"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/identity"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/volume/v3/5d5ec8450c0941d18d63558baab54737"}]}] Aug 27 02:34:32.052028 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-Tenant: tempest-AttachVolumeNegativeTest-1988669364 Aug 27 02:34:32.052028 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-Tenant-Id: 5d5ec8450c0941d18d63558baab54737 Aug 27 02:34:32.052028 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-1988669364 Aug 27 02:34:32.052028 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-User: tempest-AttachVolumeNegativeTest-1988669364-project Aug 27 02:34:32.052028 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-User-Domain-Id: default Aug 27 02:34:32.052028 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-User-Domain-Name: Default Aug 27 02:34:32.052028 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-User-Id: 26a86351583d49d090b3db09f127d49f Aug 27 02:34:32.052028 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-User-Name: tempest-AttachVolumeNegativeTest-1988669364-project Aug 27 02:34:32.052028 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: method: > Aug 27 02:34:32.052028 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: exception: Instance 77068029-afdb-4560-8d56-76c56b85ef3a could not be found. Aug 27 02:34:32.052028 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: action_args: {'id': '77068029-afdb-4560-8d56-76c56b85ef3a'} Aug 27 02:34:32.052994 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Aug 27 02:34:32.052994 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 27 02:34:32.052994 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Aug 27 02:34:32.052994 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Aug 27 02:34:32.052994 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Aug 27 02:34:32.052994 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Aug 27 02:34:32.052994 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Aug 27 02:34:32.052994 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Aug 27 02:34:32.052994 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Aug 27 02:34:32.052994 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Aug 27 02:34:32.052994 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 77068029-afdb-4560-8d56-76c56b85ef3a could not be found. Aug 27 02:34:32.052994 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi Aug 27 02:34:32.052994 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 27 02:34:32.052994 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi Aug 27 02:34:32.052994 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 27 02:34:32.052994 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 27 02:34:32.052994 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 27 02:34:32.052994 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Aug 27 02:34:32.052994 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Aug 27 02:34:32.052994 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Aug 27 02:34:32.054493 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Aug 27 02:34:32.054493 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Aug 27 02:34:32.054493 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Aug 27 02:34:32.054493 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Aug 27 02:34:32.054493 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Aug 27 02:34:32.054493 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Aug 27 02:34:32.054493 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Aug 27 02:34:32.054493 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Aug 27 02:34:32.054493 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Aug 27 02:34:32.054493 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 77068029-afdb-4560-8d56-76c56b85ef3a could not be found. Aug 27 02:34:32.054493 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi Aug 27 02:34:32.054493 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 27 02:34:32.054493 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi Aug 27 02:34:32.054493 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 27 02:34:32.054493 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 27 02:34:32.054493 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 27 02:34:32.054493 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Aug 27 02:34:32.054493 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Aug 27 02:34:32.054493 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 77068029-afdb-4560-8d56-76c56b85ef3a could not be found. Aug 27 02:34:32.054493 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi Aug 27 02:34:32.056069 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-9bb2e782-dfcc-4c2f-afa8-05b4d948b8db tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Returning 404 to user: Instance 77068029-afdb-4560-8d56-76c56b85ef3a could not be found. {{(pid=120864) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Aug 27 02:34:32.056069 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-9bb2e782-dfcc-4c2f-afa8-05b4d948b8db tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] 10.222.0.52 "DELETE /compute/v2.1/servers/77068029-afdb-4560-8d56-76c56b85ef3a" status: 404 len: 111 microversion: 2.1 time: 0.032654 Aug 27 02:34:32.056069 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 508/1009] 10.222.0.52 () {58 vars in 1239 bytes} [Sat Aug 27 02:34:32 2022] DELETE /compute/v2.1/servers/77068029-afdb-4560-8d56-76c56b85ef3a => generated 111 bytes in 34 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Aug 27 02:34:32.062397 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-7d07e7a8-2069-4053-bdf8-345415d3ace2 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:34:32.062989 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-7d07e7a8-2069-4053-bdf8-345415d3ace2 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] [instance: 77068029-afdb-4560-8d56-76c56b85ef3a] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:34:32.070030 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-7d07e7a8-2069-4053-bdf8-345415d3ace2 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:34:32.070518 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-7d07e7a8-2069-4053-bdf8-345415d3ace2 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:34:32.070953 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-7d07e7a8-2069-4053-bdf8-345415d3ace2 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:34:32.071648 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-7d07e7a8-2069-4053-bdf8-345415d3ace2 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:34:32.072105 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-7d07e7a8-2069-4053-bdf8-345415d3ace2 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:34:32.072524 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-7d07e7a8-2069-4053-bdf8-345415d3ace2 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:34:32.086698 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.wsgi [None req-7d07e7a8-2069-4053-bdf8-345415d3ace2 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] HTTP exception thrown: Instance 77068029-afdb-4560-8d56-76c56b85ef3a could not be found. Aug 27 02:34:32.088376 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-7d07e7a8-2069-4053-bdf8-345415d3ace2 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Request method failure captured: Aug 27 02:34:32.088376 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: request: GET /compute/v2.1/servers/77068029-afdb-4560-8d56-76c56b85ef3a HTTP/1.1 Aug 27 02:34:32.088376 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: Accept: application/json Aug 27 02:34:32.088376 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: Accept-Encoding: identity Aug 27 02:34:32.088376 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: Connection: close Aug 27 02:34:32.088376 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: Content-Length: 0 Aug 27 02:34:32.088376 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: Content-Type: application/json Aug 27 02:34:32.088376 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: Host: 10.222.0.52 Aug 27 02:34:32.088376 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: Openstack-System-Scope: None Aug 27 02:34:32.088376 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: User-Agent: python-urllib3/1.26.11 Aug 27 02:34:32.088376 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-Auth-Token: gAAAAABjCX_pkOb4-Fffq6Hbe0cfjlj1ARL1JmORSzrYWucWmWgZ67LEorDD3IWpPkj3UP534US8GW_nja0iAKwcdxSx3e5teQzZMdrCvOvA5aSGJO8iodOjhfOIsl8L0d8om8kj8_3q-8uTHxKupSTnIQy_J-f1kC4e22VNejIIiExDsGitats Aug 27 02:34:32.088376 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-Domain-Id: None Aug 27 02:34:32.088376 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-Domain-Name: None Aug 27 02:34:32.088376 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-Identity-Status: Confirmed Aug 27 02:34:32.088376 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-Is-Admin-Project: True Aug 27 02:34:32.088376 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-Project-Domain-Id: default Aug 27 02:34:32.088376 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-Project-Domain-Name: Default Aug 27 02:34:32.088376 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-Project-Id: 5d5ec8450c0941d18d63558baab54737 Aug 27 02:34:32.088376 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-Project-Name: tempest-AttachVolumeNegativeTest-1988669364 Aug 27 02:34:32.088376 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-Role: reader,member Aug 27 02:34:32.088376 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-Roles: reader,member Aug 27 02:34:32.090512 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-Service-Catalog: [{"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.52:8080", "publicURL": "http://10.222.0.52:8080/v1/AUTH_5d5ec8450c0941d18d63558baab54737"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52:9696/networking"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/compute/v2/5d5ec8450c0941d18d63558baab54737"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/image"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/volume/v3/5d5ec8450c0941d18d63558baab54737"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/identity"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/volume/v3/5d5ec8450c0941d18d63558baab54737"}]}] Aug 27 02:34:32.090512 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-Tenant: tempest-AttachVolumeNegativeTest-1988669364 Aug 27 02:34:32.090512 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-Tenant-Id: 5d5ec8450c0941d18d63558baab54737 Aug 27 02:34:32.090512 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-1988669364 Aug 27 02:34:32.090512 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-User: tempest-AttachVolumeNegativeTest-1988669364-project Aug 27 02:34:32.090512 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-User-Domain-Id: default Aug 27 02:34:32.090512 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-User-Domain-Name: Default Aug 27 02:34:32.090512 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-User-Id: 26a86351583d49d090b3db09f127d49f Aug 27 02:34:32.090512 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-User-Name: tempest-AttachVolumeNegativeTest-1988669364-project Aug 27 02:34:32.090512 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: method: > Aug 27 02:34:32.090512 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: exception: Instance 77068029-afdb-4560-8d56-76c56b85ef3a could not be found. Aug 27 02:34:32.090512 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: action_args: {'id': '77068029-afdb-4560-8d56-76c56b85ef3a'} Aug 27 02:34:32.092721 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Aug 27 02:34:32.092721 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 27 02:34:32.092721 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Aug 27 02:34:32.092721 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Aug 27 02:34:32.092721 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Aug 27 02:34:32.092721 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Aug 27 02:34:32.092721 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Aug 27 02:34:32.092721 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Aug 27 02:34:32.092721 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Aug 27 02:34:32.092721 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Aug 27 02:34:32.092721 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 77068029-afdb-4560-8d56-76c56b85ef3a could not be found. Aug 27 02:34:32.092721 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi Aug 27 02:34:32.092721 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 27 02:34:32.092721 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi Aug 27 02:34:32.092721 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 27 02:34:32.092721 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 27 02:34:32.092721 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 27 02:34:32.092721 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Aug 27 02:34:32.092721 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Aug 27 02:34:32.092721 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Aug 27 02:34:32.095261 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Aug 27 02:34:32.095261 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Aug 27 02:34:32.095261 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi instance = self._get_server( Aug 27 02:34:32.095261 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Aug 27 02:34:32.095261 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Aug 27 02:34:32.095261 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Aug 27 02:34:32.095261 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Aug 27 02:34:32.095261 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 77068029-afdb-4560-8d56-76c56b85ef3a could not be found. Aug 27 02:34:32.095261 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi Aug 27 02:34:32.095261 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 27 02:34:32.095261 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi Aug 27 02:34:32.095261 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 27 02:34:32.095261 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 27 02:34:32.095261 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 27 02:34:32.095261 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Aug 27 02:34:32.095261 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Aug 27 02:34:32.095261 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 77068029-afdb-4560-8d56-76c56b85ef3a could not be found. Aug 27 02:34:32.095261 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi Aug 27 02:34:32.098174 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-7d07e7a8-2069-4053-bdf8-345415d3ace2 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Returning 404 to user: Instance 77068029-afdb-4560-8d56-76c56b85ef3a could not be found. {{(pid=120863) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Aug 27 02:34:32.098174 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-7d07e7a8-2069-4053-bdf8-345415d3ace2 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] 10.222.0.52 "GET /compute/v2.1/servers/77068029-afdb-4560-8d56-76c56b85ef3a" status: 404 len: 111 microversion: 2.1 time: 0.032719 Aug 27 02:34:32.098174 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 502/1010] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:34:32 2022] GET /compute/v2.1/servers/77068029-afdb-4560-8d56-76c56b85ef3a => generated 111 bytes in 34 msecs (HTTP/1.1 404) 9 headers in 379 bytes (2 switches on core 0) Aug 27 02:34:33.254483 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-2f54efdf-38d3-4599-a998-336431ed70a8 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Calling method '>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:34:33.255267 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-2f54efdf-38d3-4599-a998-336431ed70a8 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] [instance: 2ad7bd6d-558b-4b12-a22f-67f33975e5fd] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:34:33.265908 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-2f54efdf-38d3-4599-a998-336431ed70a8 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:34:33.266392 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-2f54efdf-38d3-4599-a998-336431ed70a8 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:34:33.266827 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-2f54efdf-38d3-4599-a998-336431ed70a8 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:34:33.267529 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-2f54efdf-38d3-4599-a998-336431ed70a8 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:34:33.267970 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-2f54efdf-38d3-4599-a998-336431ed70a8 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:34:33.268390 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-2f54efdf-38d3-4599-a998-336431ed70a8 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:34:33.282995 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.wsgi [None req-2f54efdf-38d3-4599-a998-336431ed70a8 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] HTTP exception thrown: Instance 2ad7bd6d-558b-4b12-a22f-67f33975e5fd could not be found. Aug 27 02:34:33.284665 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-2f54efdf-38d3-4599-a998-336431ed70a8 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Request method failure captured: Aug 27 02:34:33.284665 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: request: DELETE /compute/v2.1/servers/2ad7bd6d-558b-4b12-a22f-67f33975e5fd HTTP/1.1 Aug 27 02:34:33.284665 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: Accept: application/json Aug 27 02:34:33.284665 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: Accept-Encoding: identity Aug 27 02:34:33.284665 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: Connection: close Aug 27 02:34:33.284665 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: Content-Length: 0 Aug 27 02:34:33.284665 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: Content-Type: application/json Aug 27 02:34:33.284665 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: Host: 10.222.0.52 Aug 27 02:34:33.284665 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: Openstack-System-Scope: None Aug 27 02:34:33.284665 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: User-Agent: python-urllib3/1.26.11 Aug 27 02:34:33.284665 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-Auth-Token: gAAAAABjCX_pkOb4-Fffq6Hbe0cfjlj1ARL1JmORSzrYWucWmWgZ67LEorDD3IWpPkj3UP534US8GW_nja0iAKwcdxSx3e5teQzZMdrCvOvA5aSGJO8iodOjhfOIsl8L0d8om8kj8_3q-8uTHxKupSTnIQy_J-f1kC4e22VNejIIiExDsGitats Aug 27 02:34:33.284665 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-Domain-Id: None Aug 27 02:34:33.284665 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-Domain-Name: None Aug 27 02:34:33.284665 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-Identity-Status: Confirmed Aug 27 02:34:33.284665 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-Is-Admin-Project: True Aug 27 02:34:33.284665 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-Project-Domain-Id: default Aug 27 02:34:33.284665 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-Project-Domain-Name: Default Aug 27 02:34:33.284665 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-Project-Id: 5d5ec8450c0941d18d63558baab54737 Aug 27 02:34:33.284665 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-Project-Name: tempest-AttachVolumeNegativeTest-1988669364 Aug 27 02:34:33.284665 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-Role: reader,member Aug 27 02:34:33.284665 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-Roles: reader,member Aug 27 02:34:33.286076 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-Service-Catalog: [{"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.52:8080", "publicURL": "http://10.222.0.52:8080/v1/AUTH_5d5ec8450c0941d18d63558baab54737"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52:9696/networking"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/compute/v2/5d5ec8450c0941d18d63558baab54737"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/image"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/volume/v3/5d5ec8450c0941d18d63558baab54737"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/identity"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/volume/v3/5d5ec8450c0941d18d63558baab54737"}]}] Aug 27 02:34:33.286076 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-Tenant: tempest-AttachVolumeNegativeTest-1988669364 Aug 27 02:34:33.286076 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-Tenant-Id: 5d5ec8450c0941d18d63558baab54737 Aug 27 02:34:33.286076 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-1988669364 Aug 27 02:34:33.286076 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-User: tempest-AttachVolumeNegativeTest-1988669364-project Aug 27 02:34:33.286076 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-User-Domain-Id: default Aug 27 02:34:33.286076 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-User-Domain-Name: Default Aug 27 02:34:33.286076 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-User-Id: 26a86351583d49d090b3db09f127d49f Aug 27 02:34:33.286076 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-User-Name: tempest-AttachVolumeNegativeTest-1988669364-project Aug 27 02:34:33.286076 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: method: > Aug 27 02:34:33.286076 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: exception: Instance 2ad7bd6d-558b-4b12-a22f-67f33975e5fd could not be found. Aug 27 02:34:33.286076 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: action_args: {'id': '2ad7bd6d-558b-4b12-a22f-67f33975e5fd'} Aug 27 02:34:33.286839 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Aug 27 02:34:33.286839 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 27 02:34:33.286839 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Aug 27 02:34:33.286839 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Aug 27 02:34:33.286839 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Aug 27 02:34:33.286839 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Aug 27 02:34:33.286839 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Aug 27 02:34:33.286839 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Aug 27 02:34:33.286839 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Aug 27 02:34:33.286839 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Aug 27 02:34:33.286839 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 2ad7bd6d-558b-4b12-a22f-67f33975e5fd could not be found. Aug 27 02:34:33.286839 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi Aug 27 02:34:33.286839 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 27 02:34:33.286839 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi Aug 27 02:34:33.286839 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 27 02:34:33.286839 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 27 02:34:33.286839 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 27 02:34:33.286839 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Aug 27 02:34:33.286839 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Aug 27 02:34:33.286839 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Aug 27 02:34:33.288148 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Aug 27 02:34:33.288148 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Aug 27 02:34:33.288148 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Aug 27 02:34:33.288148 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Aug 27 02:34:33.288148 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Aug 27 02:34:33.288148 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Aug 27 02:34:33.288148 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Aug 27 02:34:33.288148 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Aug 27 02:34:33.288148 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Aug 27 02:34:33.288148 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 2ad7bd6d-558b-4b12-a22f-67f33975e5fd could not be found. Aug 27 02:34:33.288148 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi Aug 27 02:34:33.288148 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 27 02:34:33.288148 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi Aug 27 02:34:33.288148 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 27 02:34:33.288148 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 27 02:34:33.288148 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 27 02:34:33.288148 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Aug 27 02:34:33.288148 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Aug 27 02:34:33.288148 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 2ad7bd6d-558b-4b12-a22f-67f33975e5fd could not be found. Aug 27 02:34:33.288148 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi Aug 27 02:34:33.289605 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-2f54efdf-38d3-4599-a998-336431ed70a8 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Returning 404 to user: Instance 2ad7bd6d-558b-4b12-a22f-67f33975e5fd could not be found. {{(pid=120864) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Aug 27 02:34:33.289605 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-2f54efdf-38d3-4599-a998-336431ed70a8 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] 10.222.0.52 "DELETE /compute/v2.1/servers/2ad7bd6d-558b-4b12-a22f-67f33975e5fd" status: 404 len: 111 microversion: 2.1 time: 0.038079 Aug 27 02:34:33.289605 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 509/1011] 10.222.0.52 () {58 vars in 1239 bytes} [Sat Aug 27 02:34:33 2022] DELETE /compute/v2.1/servers/2ad7bd6d-558b-4b12-a22f-67f33975e5fd => generated 111 bytes in 40 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Aug 27 02:34:33.296723 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-2ec46ce9-3cc9-4b29-9397-4aea815fd221 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:34:33.297377 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-2ec46ce9-3cc9-4b29-9397-4aea815fd221 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] [instance: 2ad7bd6d-558b-4b12-a22f-67f33975e5fd] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:34:33.304073 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-2ec46ce9-3cc9-4b29-9397-4aea815fd221 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:34:33.304564 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-2ec46ce9-3cc9-4b29-9397-4aea815fd221 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:34:33.304999 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-2ec46ce9-3cc9-4b29-9397-4aea815fd221 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:34:33.305727 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-2ec46ce9-3cc9-4b29-9397-4aea815fd221 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:34:33.306182 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-2ec46ce9-3cc9-4b29-9397-4aea815fd221 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:34:33.306610 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-2ec46ce9-3cc9-4b29-9397-4aea815fd221 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:34:33.330877 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.wsgi [None req-2ec46ce9-3cc9-4b29-9397-4aea815fd221 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] HTTP exception thrown: Instance 2ad7bd6d-558b-4b12-a22f-67f33975e5fd could not be found. Aug 27 02:34:33.332480 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-2ec46ce9-3cc9-4b29-9397-4aea815fd221 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Request method failure captured: Aug 27 02:34:33.332480 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: request: GET /compute/v2.1/servers/2ad7bd6d-558b-4b12-a22f-67f33975e5fd HTTP/1.1 Aug 27 02:34:33.332480 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: Accept: application/json Aug 27 02:34:33.332480 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: Accept-Encoding: identity Aug 27 02:34:33.332480 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: Connection: close Aug 27 02:34:33.332480 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: Content-Length: 0 Aug 27 02:34:33.332480 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: Content-Type: application/json Aug 27 02:34:33.332480 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: Host: 10.222.0.52 Aug 27 02:34:33.332480 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: Openstack-System-Scope: None Aug 27 02:34:33.332480 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: User-Agent: python-urllib3/1.26.11 Aug 27 02:34:33.332480 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-Auth-Token: gAAAAABjCX_pkOb4-Fffq6Hbe0cfjlj1ARL1JmORSzrYWucWmWgZ67LEorDD3IWpPkj3UP534US8GW_nja0iAKwcdxSx3e5teQzZMdrCvOvA5aSGJO8iodOjhfOIsl8L0d8om8kj8_3q-8uTHxKupSTnIQy_J-f1kC4e22VNejIIiExDsGitats Aug 27 02:34:33.332480 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-Domain-Id: None Aug 27 02:34:33.332480 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-Domain-Name: None Aug 27 02:34:33.332480 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-Identity-Status: Confirmed Aug 27 02:34:33.332480 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-Is-Admin-Project: True Aug 27 02:34:33.332480 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-Project-Domain-Id: default Aug 27 02:34:33.332480 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-Project-Domain-Name: Default Aug 27 02:34:33.332480 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-Project-Id: 5d5ec8450c0941d18d63558baab54737 Aug 27 02:34:33.332480 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-Project-Name: tempest-AttachVolumeNegativeTest-1988669364 Aug 27 02:34:33.332480 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-Role: reader,member Aug 27 02:34:33.332480 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-Roles: reader,member Aug 27 02:34:33.334150 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-Service-Catalog: [{"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.52:8080", "publicURL": "http://10.222.0.52:8080/v1/AUTH_5d5ec8450c0941d18d63558baab54737"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52:9696/networking"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/compute/v2/5d5ec8450c0941d18d63558baab54737"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/image"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/volume/v3/5d5ec8450c0941d18d63558baab54737"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/identity"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/volume/v3/5d5ec8450c0941d18d63558baab54737"}]}] Aug 27 02:34:33.334150 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-Tenant: tempest-AttachVolumeNegativeTest-1988669364 Aug 27 02:34:33.334150 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-Tenant-Id: 5d5ec8450c0941d18d63558baab54737 Aug 27 02:34:33.334150 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-1988669364 Aug 27 02:34:33.334150 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-User: tempest-AttachVolumeNegativeTest-1988669364-project Aug 27 02:34:33.334150 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-User-Domain-Id: default Aug 27 02:34:33.334150 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-User-Domain-Name: Default Aug 27 02:34:33.334150 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-User-Id: 26a86351583d49d090b3db09f127d49f Aug 27 02:34:33.334150 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-User-Name: tempest-AttachVolumeNegativeTest-1988669364-project Aug 27 02:34:33.334150 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: method: > Aug 27 02:34:33.334150 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: exception: Instance 2ad7bd6d-558b-4b12-a22f-67f33975e5fd could not be found. Aug 27 02:34:33.334150 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: action_args: {'id': '2ad7bd6d-558b-4b12-a22f-67f33975e5fd'} Aug 27 02:34:33.334975 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Aug 27 02:34:33.334975 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 27 02:34:33.334975 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Aug 27 02:34:33.334975 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Aug 27 02:34:33.334975 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Aug 27 02:34:33.334975 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Aug 27 02:34:33.334975 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Aug 27 02:34:33.334975 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Aug 27 02:34:33.334975 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Aug 27 02:34:33.334975 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Aug 27 02:34:33.334975 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 2ad7bd6d-558b-4b12-a22f-67f33975e5fd could not be found. Aug 27 02:34:33.334975 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi Aug 27 02:34:33.334975 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 27 02:34:33.334975 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi Aug 27 02:34:33.334975 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 27 02:34:33.334975 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 27 02:34:33.334975 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 27 02:34:33.334975 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Aug 27 02:34:33.334975 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Aug 27 02:34:33.334975 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Aug 27 02:34:33.336191 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Aug 27 02:34:33.336191 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Aug 27 02:34:33.336191 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi instance = self._get_server( Aug 27 02:34:33.336191 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Aug 27 02:34:33.336191 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Aug 27 02:34:33.336191 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Aug 27 02:34:33.336191 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Aug 27 02:34:33.336191 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 2ad7bd6d-558b-4b12-a22f-67f33975e5fd could not be found. Aug 27 02:34:33.336191 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi Aug 27 02:34:33.336191 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 27 02:34:33.336191 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi Aug 27 02:34:33.336191 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 27 02:34:33.336191 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 27 02:34:33.336191 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 27 02:34:33.336191 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Aug 27 02:34:33.336191 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Aug 27 02:34:33.336191 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 2ad7bd6d-558b-4b12-a22f-67f33975e5fd could not be found. Aug 27 02:34:33.336191 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi Aug 27 02:34:33.337394 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-2ec46ce9-3cc9-4b29-9397-4aea815fd221 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Returning 404 to user: Instance 2ad7bd6d-558b-4b12-a22f-67f33975e5fd could not be found. {{(pid=120863) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Aug 27 02:34:33.337394 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-2ec46ce9-3cc9-4b29-9397-4aea815fd221 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] 10.222.0.52 "GET /compute/v2.1/servers/2ad7bd6d-558b-4b12-a22f-67f33975e5fd" status: 404 len: 111 microversion: 2.1 time: 0.041503 Aug 27 02:34:33.337394 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 503/1012] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:34:33 2022] GET /compute/v2.1/servers/2ad7bd6d-558b-4b12-a22f-67f33975e5fd => generated 111 bytes in 43 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Aug 27 02:34:33.344549 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-e212dee0-3521-4245-8949-8cd8dbe07007 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Calling method '>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:34:33.345044 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-e212dee0-3521-4245-8949-8cd8dbe07007 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] [instance: dc699df0-be4f-4036-9e01-45ba0156616b] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:34:33.351715 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-e212dee0-3521-4245-8949-8cd8dbe07007 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:34:33.352206 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-e212dee0-3521-4245-8949-8cd8dbe07007 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:34:33.352636 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-e212dee0-3521-4245-8949-8cd8dbe07007 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:34:33.353483 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-e212dee0-3521-4245-8949-8cd8dbe07007 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:34:33.353930 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-e212dee0-3521-4245-8949-8cd8dbe07007 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:34:33.354356 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-e212dee0-3521-4245-8949-8cd8dbe07007 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:34:33.369308 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.wsgi [None req-e212dee0-3521-4245-8949-8cd8dbe07007 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] HTTP exception thrown: Instance dc699df0-be4f-4036-9e01-45ba0156616b could not be found. Aug 27 02:34:33.371033 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-e212dee0-3521-4245-8949-8cd8dbe07007 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Request method failure captured: Aug 27 02:34:33.371033 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: request: DELETE /compute/v2.1/servers/dc699df0-be4f-4036-9e01-45ba0156616b HTTP/1.1 Aug 27 02:34:33.371033 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: Accept: application/json Aug 27 02:34:33.371033 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: Accept-Encoding: identity Aug 27 02:34:33.371033 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: Connection: close Aug 27 02:34:33.371033 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: Content-Length: 0 Aug 27 02:34:33.371033 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: Content-Type: application/json Aug 27 02:34:33.371033 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: Host: 10.222.0.52 Aug 27 02:34:33.371033 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: Openstack-System-Scope: None Aug 27 02:34:33.371033 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: User-Agent: python-urllib3/1.26.11 Aug 27 02:34:33.371033 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-Auth-Token: gAAAAABjCX_pkOb4-Fffq6Hbe0cfjlj1ARL1JmORSzrYWucWmWgZ67LEorDD3IWpPkj3UP534US8GW_nja0iAKwcdxSx3e5teQzZMdrCvOvA5aSGJO8iodOjhfOIsl8L0d8om8kj8_3q-8uTHxKupSTnIQy_J-f1kC4e22VNejIIiExDsGitats Aug 27 02:34:33.371033 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-Domain-Id: None Aug 27 02:34:33.371033 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-Domain-Name: None Aug 27 02:34:33.371033 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-Identity-Status: Confirmed Aug 27 02:34:33.371033 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-Is-Admin-Project: True Aug 27 02:34:33.371033 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-Project-Domain-Id: default Aug 27 02:34:33.371033 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-Project-Domain-Name: Default Aug 27 02:34:33.371033 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-Project-Id: 5d5ec8450c0941d18d63558baab54737 Aug 27 02:34:33.371033 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-Project-Name: tempest-AttachVolumeNegativeTest-1988669364 Aug 27 02:34:33.371033 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-Role: reader,member Aug 27 02:34:33.371033 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-Roles: reader,member Aug 27 02:34:33.372652 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-Service-Catalog: [{"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.52:8080", "publicURL": "http://10.222.0.52:8080/v1/AUTH_5d5ec8450c0941d18d63558baab54737"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52:9696/networking"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/compute/v2/5d5ec8450c0941d18d63558baab54737"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/image"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/volume/v3/5d5ec8450c0941d18d63558baab54737"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/identity"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/volume/v3/5d5ec8450c0941d18d63558baab54737"}]}] Aug 27 02:34:33.372652 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-Tenant: tempest-AttachVolumeNegativeTest-1988669364 Aug 27 02:34:33.372652 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-Tenant-Id: 5d5ec8450c0941d18d63558baab54737 Aug 27 02:34:33.372652 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-1988669364 Aug 27 02:34:33.372652 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-User: tempest-AttachVolumeNegativeTest-1988669364-project Aug 27 02:34:33.372652 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-User-Domain-Id: default Aug 27 02:34:33.372652 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-User-Domain-Name: Default Aug 27 02:34:33.372652 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-User-Id: 26a86351583d49d090b3db09f127d49f Aug 27 02:34:33.372652 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-User-Name: tempest-AttachVolumeNegativeTest-1988669364-project Aug 27 02:34:33.372652 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: method: > Aug 27 02:34:33.372652 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: exception: Instance dc699df0-be4f-4036-9e01-45ba0156616b could not be found. Aug 27 02:34:33.372652 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: action_args: {'id': 'dc699df0-be4f-4036-9e01-45ba0156616b'} Aug 27 02:34:33.373610 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Aug 27 02:34:33.373610 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 27 02:34:33.373610 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Aug 27 02:34:33.373610 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Aug 27 02:34:33.373610 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Aug 27 02:34:33.373610 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Aug 27 02:34:33.373610 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Aug 27 02:34:33.373610 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Aug 27 02:34:33.373610 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Aug 27 02:34:33.373610 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Aug 27 02:34:33.373610 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance dc699df0-be4f-4036-9e01-45ba0156616b could not be found. Aug 27 02:34:33.373610 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi Aug 27 02:34:33.373610 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 27 02:34:33.373610 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi Aug 27 02:34:33.373610 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 27 02:34:33.373610 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 27 02:34:33.373610 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 27 02:34:33.373610 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Aug 27 02:34:33.373610 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Aug 27 02:34:33.373610 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Aug 27 02:34:33.374851 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Aug 27 02:34:33.374851 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Aug 27 02:34:33.374851 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Aug 27 02:34:33.374851 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Aug 27 02:34:33.374851 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Aug 27 02:34:33.374851 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Aug 27 02:34:33.374851 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Aug 27 02:34:33.374851 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Aug 27 02:34:33.374851 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Aug 27 02:34:33.374851 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance dc699df0-be4f-4036-9e01-45ba0156616b could not be found. Aug 27 02:34:33.374851 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi Aug 27 02:34:33.374851 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 27 02:34:33.374851 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi Aug 27 02:34:33.374851 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 27 02:34:33.374851 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 27 02:34:33.374851 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 27 02:34:33.374851 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Aug 27 02:34:33.374851 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Aug 27 02:34:33.374851 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance dc699df0-be4f-4036-9e01-45ba0156616b could not be found. Aug 27 02:34:33.374851 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi Aug 27 02:34:33.376098 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-e212dee0-3521-4245-8949-8cd8dbe07007 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Returning 404 to user: Instance dc699df0-be4f-4036-9e01-45ba0156616b could not be found. {{(pid=120864) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Aug 27 02:34:33.376098 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-e212dee0-3521-4245-8949-8cd8dbe07007 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] 10.222.0.52 "DELETE /compute/v2.1/servers/dc699df0-be4f-4036-9e01-45ba0156616b" status: 404 len: 111 microversion: 2.1 time: 0.033268 Aug 27 02:34:33.376098 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 510/1013] 10.222.0.52 () {58 vars in 1239 bytes} [Sat Aug 27 02:34:33 2022] DELETE /compute/v2.1/servers/dc699df0-be4f-4036-9e01-45ba0156616b => generated 111 bytes in 35 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Aug 27 02:34:33.381778 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-fdcd83a3-5820-4317-8e7f-6001d9bddf14 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:34:33.382393 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-fdcd83a3-5820-4317-8e7f-6001d9bddf14 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] [instance: dc699df0-be4f-4036-9e01-45ba0156616b] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:34:33.389676 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-fdcd83a3-5820-4317-8e7f-6001d9bddf14 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:34:33.390164 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-fdcd83a3-5820-4317-8e7f-6001d9bddf14 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:34:33.390597 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-fdcd83a3-5820-4317-8e7f-6001d9bddf14 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:34:33.391293 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-fdcd83a3-5820-4317-8e7f-6001d9bddf14 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:34:33.391744 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-fdcd83a3-5820-4317-8e7f-6001d9bddf14 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:34:33.392167 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-fdcd83a3-5820-4317-8e7f-6001d9bddf14 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:34:33.406762 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.wsgi [None req-fdcd83a3-5820-4317-8e7f-6001d9bddf14 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] HTTP exception thrown: Instance dc699df0-be4f-4036-9e01-45ba0156616b could not be found. Aug 27 02:34:33.408361 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-fdcd83a3-5820-4317-8e7f-6001d9bddf14 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Request method failure captured: Aug 27 02:34:33.408361 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: request: GET /compute/v2.1/servers/dc699df0-be4f-4036-9e01-45ba0156616b HTTP/1.1 Aug 27 02:34:33.408361 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: Accept: application/json Aug 27 02:34:33.408361 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: Accept-Encoding: identity Aug 27 02:34:33.408361 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: Connection: close Aug 27 02:34:33.408361 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: Content-Length: 0 Aug 27 02:34:33.408361 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: Content-Type: application/json Aug 27 02:34:33.408361 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: Host: 10.222.0.52 Aug 27 02:34:33.408361 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: Openstack-System-Scope: None Aug 27 02:34:33.408361 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: User-Agent: python-urllib3/1.26.11 Aug 27 02:34:33.408361 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-Auth-Token: gAAAAABjCX_pkOb4-Fffq6Hbe0cfjlj1ARL1JmORSzrYWucWmWgZ67LEorDD3IWpPkj3UP534US8GW_nja0iAKwcdxSx3e5teQzZMdrCvOvA5aSGJO8iodOjhfOIsl8L0d8om8kj8_3q-8uTHxKupSTnIQy_J-f1kC4e22VNejIIiExDsGitats Aug 27 02:34:33.408361 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-Domain-Id: None Aug 27 02:34:33.408361 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-Domain-Name: None Aug 27 02:34:33.408361 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-Identity-Status: Confirmed Aug 27 02:34:33.408361 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-Is-Admin-Project: True Aug 27 02:34:33.408361 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-Project-Domain-Id: default Aug 27 02:34:33.408361 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-Project-Domain-Name: Default Aug 27 02:34:33.408361 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-Project-Id: 5d5ec8450c0941d18d63558baab54737 Aug 27 02:34:33.408361 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-Project-Name: tempest-AttachVolumeNegativeTest-1988669364 Aug 27 02:34:33.408361 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-Role: reader,member Aug 27 02:34:33.408361 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-Roles: reader,member Aug 27 02:34:33.409985 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-Service-Catalog: [{"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.52:8080", "publicURL": "http://10.222.0.52:8080/v1/AUTH_5d5ec8450c0941d18d63558baab54737"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52:9696/networking"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/compute/v2/5d5ec8450c0941d18d63558baab54737"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/image"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/volume/v3/5d5ec8450c0941d18d63558baab54737"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/identity"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/volume/v3/5d5ec8450c0941d18d63558baab54737"}]}] Aug 27 02:34:33.409985 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-Tenant: tempest-AttachVolumeNegativeTest-1988669364 Aug 27 02:34:33.409985 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-Tenant-Id: 5d5ec8450c0941d18d63558baab54737 Aug 27 02:34:33.409985 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-1988669364 Aug 27 02:34:33.409985 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-User: tempest-AttachVolumeNegativeTest-1988669364-project Aug 27 02:34:33.409985 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-User-Domain-Id: default Aug 27 02:34:33.409985 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-User-Domain-Name: Default Aug 27 02:34:33.409985 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-User-Id: 26a86351583d49d090b3db09f127d49f Aug 27 02:34:33.409985 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-User-Name: tempest-AttachVolumeNegativeTest-1988669364-project Aug 27 02:34:33.409985 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: method: > Aug 27 02:34:33.409985 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: exception: Instance dc699df0-be4f-4036-9e01-45ba0156616b could not be found. Aug 27 02:34:33.409985 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: action_args: {'id': 'dc699df0-be4f-4036-9e01-45ba0156616b'} Aug 27 02:34:33.410776 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Aug 27 02:34:33.410776 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 27 02:34:33.410776 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Aug 27 02:34:33.410776 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Aug 27 02:34:33.410776 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Aug 27 02:34:33.410776 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Aug 27 02:34:33.410776 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Aug 27 02:34:33.410776 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Aug 27 02:34:33.410776 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Aug 27 02:34:33.410776 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Aug 27 02:34:33.410776 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance dc699df0-be4f-4036-9e01-45ba0156616b could not be found. Aug 27 02:34:33.410776 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi Aug 27 02:34:33.410776 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 27 02:34:33.410776 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi Aug 27 02:34:33.410776 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 27 02:34:33.410776 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 27 02:34:33.410776 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 27 02:34:33.410776 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Aug 27 02:34:33.410776 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Aug 27 02:34:33.410776 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Aug 27 02:34:33.412903 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Aug 27 02:34:33.412903 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Aug 27 02:34:33.412903 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi instance = self._get_server( Aug 27 02:34:33.412903 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Aug 27 02:34:33.412903 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Aug 27 02:34:33.412903 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Aug 27 02:34:33.412903 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Aug 27 02:34:33.412903 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance dc699df0-be4f-4036-9e01-45ba0156616b could not be found. Aug 27 02:34:33.412903 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi Aug 27 02:34:33.412903 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 27 02:34:33.412903 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi Aug 27 02:34:33.412903 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 27 02:34:33.412903 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 27 02:34:33.412903 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 27 02:34:33.412903 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Aug 27 02:34:33.412903 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Aug 27 02:34:33.412903 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance dc699df0-be4f-4036-9e01-45ba0156616b could not be found. Aug 27 02:34:33.412903 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi Aug 27 02:34:33.414322 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-fdcd83a3-5820-4317-8e7f-6001d9bddf14 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Returning 404 to user: Instance dc699df0-be4f-4036-9e01-45ba0156616b could not be found. {{(pid=120863) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Aug 27 02:34:33.414322 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-fdcd83a3-5820-4317-8e7f-6001d9bddf14 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] 10.222.0.52 "GET /compute/v2.1/servers/dc699df0-be4f-4036-9e01-45ba0156616b" status: 404 len: 111 microversion: 2.1 time: 0.031973 Aug 27 02:34:33.414322 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 504/1014] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:34:33 2022] GET /compute/v2.1/servers/dc699df0-be4f-4036-9e01-45ba0156616b => generated 111 bytes in 33 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Aug 27 02:34:34.575507 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-c94f6d47-4d55-4e3c-b2b3-697af3500977 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Calling method '>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:34:34.576248 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-c94f6d47-4d55-4e3c-b2b3-697af3500977 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] [instance: 2ad09ab6-ef1f-47a2-bf44-6ae046cc8645] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:34:34.583077 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-c94f6d47-4d55-4e3c-b2b3-697af3500977 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:34:34.583567 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-c94f6d47-4d55-4e3c-b2b3-697af3500977 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:34:34.584031 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-c94f6d47-4d55-4e3c-b2b3-697af3500977 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:34:34.584724 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-c94f6d47-4d55-4e3c-b2b3-697af3500977 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:34:34.585212 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-c94f6d47-4d55-4e3c-b2b3-697af3500977 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:34:34.585634 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-c94f6d47-4d55-4e3c-b2b3-697af3500977 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:34:34.600150 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.wsgi [None req-c94f6d47-4d55-4e3c-b2b3-697af3500977 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] HTTP exception thrown: Instance 2ad09ab6-ef1f-47a2-bf44-6ae046cc8645 could not be found. Aug 27 02:34:34.602133 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-c94f6d47-4d55-4e3c-b2b3-697af3500977 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Request method failure captured: Aug 27 02:34:34.602133 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: request: DELETE /compute/v2.1/servers/2ad09ab6-ef1f-47a2-bf44-6ae046cc8645 HTTP/1.1 Aug 27 02:34:34.602133 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: Accept: application/json Aug 27 02:34:34.602133 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: Accept-Encoding: identity Aug 27 02:34:34.602133 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: Connection: close Aug 27 02:34:34.602133 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: Content-Length: 0 Aug 27 02:34:34.602133 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: Content-Type: application/json Aug 27 02:34:34.602133 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: Host: 10.222.0.52 Aug 27 02:34:34.602133 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: Openstack-System-Scope: None Aug 27 02:34:34.602133 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: User-Agent: python-urllib3/1.26.11 Aug 27 02:34:34.602133 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-Auth-Token: gAAAAABjCX_pkOb4-Fffq6Hbe0cfjlj1ARL1JmORSzrYWucWmWgZ67LEorDD3IWpPkj3UP534US8GW_nja0iAKwcdxSx3e5teQzZMdrCvOvA5aSGJO8iodOjhfOIsl8L0d8om8kj8_3q-8uTHxKupSTnIQy_J-f1kC4e22VNejIIiExDsGitats Aug 27 02:34:34.602133 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-Domain-Id: None Aug 27 02:34:34.602133 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-Domain-Name: None Aug 27 02:34:34.602133 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-Identity-Status: Confirmed Aug 27 02:34:34.602133 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-Is-Admin-Project: True Aug 27 02:34:34.602133 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-Project-Domain-Id: default Aug 27 02:34:34.602133 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-Project-Domain-Name: Default Aug 27 02:34:34.602133 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-Project-Id: 5d5ec8450c0941d18d63558baab54737 Aug 27 02:34:34.602133 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-Project-Name: tempest-AttachVolumeNegativeTest-1988669364 Aug 27 02:34:34.602133 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-Role: reader,member Aug 27 02:34:34.602133 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-Roles: reader,member Aug 27 02:34:34.603904 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-Service-Catalog: [{"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.52:8080", "publicURL": "http://10.222.0.52:8080/v1/AUTH_5d5ec8450c0941d18d63558baab54737"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52:9696/networking"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/compute/v2/5d5ec8450c0941d18d63558baab54737"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/image"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/volume/v3/5d5ec8450c0941d18d63558baab54737"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/identity"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/volume/v3/5d5ec8450c0941d18d63558baab54737"}]}] Aug 27 02:34:34.603904 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-Tenant: tempest-AttachVolumeNegativeTest-1988669364 Aug 27 02:34:34.603904 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-Tenant-Id: 5d5ec8450c0941d18d63558baab54737 Aug 27 02:34:34.603904 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-1988669364 Aug 27 02:34:34.603904 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-User: tempest-AttachVolumeNegativeTest-1988669364-project Aug 27 02:34:34.603904 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-User-Domain-Id: default Aug 27 02:34:34.603904 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-User-Domain-Name: Default Aug 27 02:34:34.603904 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-User-Id: 26a86351583d49d090b3db09f127d49f Aug 27 02:34:34.603904 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-User-Name: tempest-AttachVolumeNegativeTest-1988669364-project Aug 27 02:34:34.603904 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: method: > Aug 27 02:34:34.603904 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: exception: Instance 2ad09ab6-ef1f-47a2-bf44-6ae046cc8645 could not be found. Aug 27 02:34:34.603904 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: action_args: {'id': '2ad09ab6-ef1f-47a2-bf44-6ae046cc8645'} Aug 27 02:34:34.604778 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Aug 27 02:34:34.604778 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 27 02:34:34.604778 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Aug 27 02:34:34.604778 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Aug 27 02:34:34.604778 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Aug 27 02:34:34.604778 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Aug 27 02:34:34.604778 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Aug 27 02:34:34.604778 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Aug 27 02:34:34.604778 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Aug 27 02:34:34.604778 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Aug 27 02:34:34.604778 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 2ad09ab6-ef1f-47a2-bf44-6ae046cc8645 could not be found. Aug 27 02:34:34.604778 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi Aug 27 02:34:34.604778 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 27 02:34:34.604778 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi Aug 27 02:34:34.604778 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 27 02:34:34.604778 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 27 02:34:34.604778 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 27 02:34:34.604778 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Aug 27 02:34:34.604778 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Aug 27 02:34:34.604778 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Aug 27 02:34:34.606203 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Aug 27 02:34:34.606203 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Aug 27 02:34:34.606203 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Aug 27 02:34:34.606203 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Aug 27 02:34:34.606203 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Aug 27 02:34:34.606203 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Aug 27 02:34:34.606203 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Aug 27 02:34:34.606203 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Aug 27 02:34:34.606203 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Aug 27 02:34:34.606203 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 2ad09ab6-ef1f-47a2-bf44-6ae046cc8645 could not be found. Aug 27 02:34:34.606203 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi Aug 27 02:34:34.606203 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 27 02:34:34.606203 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi Aug 27 02:34:34.606203 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 27 02:34:34.606203 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 27 02:34:34.606203 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 27 02:34:34.606203 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Aug 27 02:34:34.606203 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Aug 27 02:34:34.606203 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 2ad09ab6-ef1f-47a2-bf44-6ae046cc8645 could not be found. Aug 27 02:34:34.606203 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi Aug 27 02:34:34.607893 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-c94f6d47-4d55-4e3c-b2b3-697af3500977 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Returning 404 to user: Instance 2ad09ab6-ef1f-47a2-bf44-6ae046cc8645 could not be found. {{(pid=120864) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Aug 27 02:34:34.607893 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-c94f6d47-4d55-4e3c-b2b3-697af3500977 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] 10.222.0.52 "DELETE /compute/v2.1/servers/2ad09ab6-ef1f-47a2-bf44-6ae046cc8645" status: 404 len: 111 microversion: 2.1 time: 0.032297 Aug 27 02:34:34.607893 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 511/1015] 10.222.0.52 () {58 vars in 1239 bytes} [Sat Aug 27 02:34:34 2022] DELETE /compute/v2.1/servers/2ad09ab6-ef1f-47a2-bf44-6ae046cc8645 => generated 111 bytes in 33 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Aug 27 02:34:34.613481 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-251936e6-c79f-4f70-97ab-64b750dad4b3 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:34:34.614144 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-251936e6-c79f-4f70-97ab-64b750dad4b3 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] [instance: 2ad09ab6-ef1f-47a2-bf44-6ae046cc8645] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:34:34.621101 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-251936e6-c79f-4f70-97ab-64b750dad4b3 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:34:34.621615 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-251936e6-c79f-4f70-97ab-64b750dad4b3 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:34:34.622058 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-251936e6-c79f-4f70-97ab-64b750dad4b3 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:34:34.622787 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-251936e6-c79f-4f70-97ab-64b750dad4b3 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:34:34.623338 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-251936e6-c79f-4f70-97ab-64b750dad4b3 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:34:34.623763 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-251936e6-c79f-4f70-97ab-64b750dad4b3 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:34:34.638098 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.wsgi [None req-251936e6-c79f-4f70-97ab-64b750dad4b3 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] HTTP exception thrown: Instance 2ad09ab6-ef1f-47a2-bf44-6ae046cc8645 could not be found. Aug 27 02:34:34.639766 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-251936e6-c79f-4f70-97ab-64b750dad4b3 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Request method failure captured: Aug 27 02:34:34.639766 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: request: GET /compute/v2.1/servers/2ad09ab6-ef1f-47a2-bf44-6ae046cc8645 HTTP/1.1 Aug 27 02:34:34.639766 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: Accept: application/json Aug 27 02:34:34.639766 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: Accept-Encoding: identity Aug 27 02:34:34.639766 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: Connection: close Aug 27 02:34:34.639766 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: Content-Length: 0 Aug 27 02:34:34.639766 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: Content-Type: application/json Aug 27 02:34:34.639766 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: Host: 10.222.0.52 Aug 27 02:34:34.639766 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: Openstack-System-Scope: None Aug 27 02:34:34.639766 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: User-Agent: python-urllib3/1.26.11 Aug 27 02:34:34.639766 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-Auth-Token: gAAAAABjCX_pkOb4-Fffq6Hbe0cfjlj1ARL1JmORSzrYWucWmWgZ67LEorDD3IWpPkj3UP534US8GW_nja0iAKwcdxSx3e5teQzZMdrCvOvA5aSGJO8iodOjhfOIsl8L0d8om8kj8_3q-8uTHxKupSTnIQy_J-f1kC4e22VNejIIiExDsGitats Aug 27 02:34:34.639766 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-Domain-Id: None Aug 27 02:34:34.639766 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-Domain-Name: None Aug 27 02:34:34.639766 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-Identity-Status: Confirmed Aug 27 02:34:34.639766 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-Is-Admin-Project: True Aug 27 02:34:34.639766 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-Project-Domain-Id: default Aug 27 02:34:34.639766 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-Project-Domain-Name: Default Aug 27 02:34:34.639766 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-Project-Id: 5d5ec8450c0941d18d63558baab54737 Aug 27 02:34:34.639766 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-Project-Name: tempest-AttachVolumeNegativeTest-1988669364 Aug 27 02:34:34.639766 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-Role: reader,member Aug 27 02:34:34.639766 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-Roles: reader,member Aug 27 02:34:34.641846 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-Service-Catalog: [{"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.52:8080", "publicURL": "http://10.222.0.52:8080/v1/AUTH_5d5ec8450c0941d18d63558baab54737"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52:9696/networking"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/compute/v2/5d5ec8450c0941d18d63558baab54737"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/image"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/volume/v3/5d5ec8450c0941d18d63558baab54737"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/identity"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/volume/v3/5d5ec8450c0941d18d63558baab54737"}]}] Aug 27 02:34:34.641846 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-Tenant: tempest-AttachVolumeNegativeTest-1988669364 Aug 27 02:34:34.641846 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-Tenant-Id: 5d5ec8450c0941d18d63558baab54737 Aug 27 02:34:34.641846 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-1988669364 Aug 27 02:34:34.641846 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-User: tempest-AttachVolumeNegativeTest-1988669364-project Aug 27 02:34:34.641846 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-User-Domain-Id: default Aug 27 02:34:34.641846 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-User-Domain-Name: Default Aug 27 02:34:34.641846 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-User-Id: 26a86351583d49d090b3db09f127d49f Aug 27 02:34:34.641846 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-User-Name: tempest-AttachVolumeNegativeTest-1988669364-project Aug 27 02:34:34.641846 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: method: > Aug 27 02:34:34.641846 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: exception: Instance 2ad09ab6-ef1f-47a2-bf44-6ae046cc8645 could not be found. Aug 27 02:34:34.641846 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: action_args: {'id': '2ad09ab6-ef1f-47a2-bf44-6ae046cc8645'} Aug 27 02:34:34.642623 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Aug 27 02:34:34.642623 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 27 02:34:34.642623 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Aug 27 02:34:34.642623 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Aug 27 02:34:34.642623 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Aug 27 02:34:34.642623 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Aug 27 02:34:34.642623 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Aug 27 02:34:34.642623 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Aug 27 02:34:34.642623 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Aug 27 02:34:34.642623 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Aug 27 02:34:34.642623 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 2ad09ab6-ef1f-47a2-bf44-6ae046cc8645 could not be found. Aug 27 02:34:34.642623 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi Aug 27 02:34:34.642623 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 27 02:34:34.642623 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi Aug 27 02:34:34.642623 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 27 02:34:34.642623 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 27 02:34:34.642623 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 27 02:34:34.642623 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Aug 27 02:34:34.642623 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Aug 27 02:34:34.642623 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Aug 27 02:34:34.643832 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Aug 27 02:34:34.643832 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Aug 27 02:34:34.643832 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi instance = self._get_server( Aug 27 02:34:34.643832 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Aug 27 02:34:34.643832 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Aug 27 02:34:34.643832 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Aug 27 02:34:34.643832 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Aug 27 02:34:34.643832 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 2ad09ab6-ef1f-47a2-bf44-6ae046cc8645 could not be found. Aug 27 02:34:34.643832 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi Aug 27 02:34:34.643832 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 27 02:34:34.643832 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi Aug 27 02:34:34.643832 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 27 02:34:34.643832 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 27 02:34:34.643832 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 27 02:34:34.643832 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Aug 27 02:34:34.643832 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Aug 27 02:34:34.643832 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 2ad09ab6-ef1f-47a2-bf44-6ae046cc8645 could not be found. Aug 27 02:34:34.643832 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi Aug 27 02:34:34.644923 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-251936e6-c79f-4f70-97ab-64b750dad4b3 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] Returning 404 to user: Instance 2ad09ab6-ef1f-47a2-bf44-6ae046cc8645 could not be found. {{(pid=120863) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Aug 27 02:34:34.644923 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-251936e6-c79f-4f70-97ab-64b750dad4b3 tempest-AttachVolumeNegativeTest-1988669364 tempest-AttachVolumeNegativeTest-1988669364-project] 10.222.0.52 "GET /compute/v2.1/servers/2ad09ab6-ef1f-47a2-bf44-6ae046cc8645" status: 404 len: 111 microversion: 2.1 time: 0.031719 Aug 27 02:34:34.644923 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 505/1016] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:34:34 2022] GET /compute/v2.1/servers/2ad09ab6-ef1f-47a2-bf44-6ae046cc8645 => generated 111 bytes in 33 msecs (HTTP/1.1 404) 9 headers in 379 bytes (2 switches on core 0) Aug 27 02:34:39.610515 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-54293fae-c6c8-427b-ad9d-28d0e3cb1bc4 tempest-VolumesNegativeTest-179747936 tempest-VolumesNegativeTest-179747936-project] Action: 'create', calling method: .version_select at 0x7f6b1e953310>, body: {"volume": {"size": "#$%", "display_name": "tempest-VolumesNegativeTest-Volume-25830687", "metadata": {"Type": "work"}}} {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 27 02:34:39.623796 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-54293fae-c6c8-427b-ad9d-28d0e3cb1bc4 tempest-VolumesNegativeTest-179747936 tempest-VolumesNegativeTest-179747936-project] Request method failure captured: Aug 27 02:34:39.623796 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: request: POST /compute/v2.1/os-volumes HTTP/1.1 Aug 27 02:34:39.623796 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: Accept: application/json Aug 27 02:34:39.623796 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: Accept-Encoding: identity Aug 27 02:34:39.623796 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: Connection: close Aug 27 02:34:39.623796 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: Content-Length: 120 Aug 27 02:34:39.623796 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: Content-Type: application/json Aug 27 02:34:39.623796 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: Host: 10.222.0.52 Aug 27 02:34:39.623796 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: Openstack-System-Scope: None Aug 27 02:34:39.623796 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: User-Agent: python-urllib3/1.26.11 Aug 27 02:34:39.623796 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-Auth-Token: gAAAAABjCYK_pglciF0SMH3Qi_0urNVub6blq0nOW-_hsh0k3tlzIs0UWRxF1356dcQqMh2udDm-HCp0Lfw30RRl9YP-HPrDCF977-mCIcasgDMbn7J6kIPOHllyhaiDe3rMrTupqETInorHI8eCV02vLsldzFsLBOSeSDQ8nVhC03wxGZpOCU0 Aug 27 02:34:39.623796 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-Domain-Id: None Aug 27 02:34:39.623796 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-Domain-Name: None Aug 27 02:34:39.623796 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-Identity-Status: Confirmed Aug 27 02:34:39.623796 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-Is-Admin-Project: True Aug 27 02:34:39.623796 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-Project-Domain-Id: default Aug 27 02:34:39.623796 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-Project-Domain-Name: Default Aug 27 02:34:39.623796 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-Project-Id: 5d046a832eac49fda2b3efd3315cda88 Aug 27 02:34:39.623796 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-Project-Name: tempest-VolumesNegativeTest-179747936 Aug 27 02:34:39.623796 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-Role: reader,member Aug 27 02:34:39.623796 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-Roles: reader,member Aug 27 02:34:39.625504 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-Service-Catalog: [{"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.52:8080", "publicURL": "http://10.222.0.52:8080/v1/AUTH_5d046a832eac49fda2b3efd3315cda88"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52:9696/networking"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/compute/v2/5d046a832eac49fda2b3efd3315cda88"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/image"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/volume/v3/5d046a832eac49fda2b3efd3315cda88"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/identity"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/volume/v3/5d046a832eac49fda2b3efd3315cda88"}]}] Aug 27 02:34:39.625504 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-Tenant: tempest-VolumesNegativeTest-179747936 Aug 27 02:34:39.625504 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-Tenant-Id: 5d046a832eac49fda2b3efd3315cda88 Aug 27 02:34:39.625504 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-Tenant-Name: tempest-VolumesNegativeTest-179747936 Aug 27 02:34:39.625504 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-User: tempest-VolumesNegativeTest-179747936-project Aug 27 02:34:39.625504 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-User-Domain-Id: default Aug 27 02:34:39.625504 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-User-Domain-Name: Default Aug 27 02:34:39.625504 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-User-Id: 03f2912b300d4db392def0d602019c65 Aug 27 02:34:39.625504 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-User-Name: tempest-VolumesNegativeTest-179747936-project Aug 27 02:34:39.625504 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: Aug 27 02:34:39.625504 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: {"volume": {"size": "#$%", "display_name": "tempest-VolumesNegativeTest-Volume-25830687", "metadata": {"Type": "work"}}} Aug 27 02:34:39.625504 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: method: Aug 27 02:34:39.625504 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: exception: Invalid input for field/attribute size. Value: #$%. '#$%' does not match '^[0-9]+$' Aug 27 02:34:39.626538 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: action_args: {'body': {'volume': {'size': '#$%', 'display_name': 'tempest-VolumesNegativeTest-Volume-25830687', 'metadata': {'Type': 'work'}}}} Aug 27 02:34:39.626538 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Aug 27 02:34:39.626538 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 27 02:34:39.626538 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/validators.py", line 302, in validate Aug 27 02:34:39.626538 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi self.validator.validate(*args, **kwargs) Aug 27 02:34:39.626538 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/jsonschema/validators.py", line 268, in validate Aug 27 02:34:39.626538 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi raise error Aug 27 02:34:39.626538 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi jsonschema.exceptions.ValidationError: '#$%' does not match '^[0-9]+$' Aug 27 02:34:39.626538 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi Aug 27 02:34:39.626538 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi Failed validating 'pattern' in schema['properties']['volume']['properties']['size']: Aug 27 02:34:39.626538 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi {'minimum': 1, 'pattern': '^[0-9]+$', 'type': ['integer', 'string']} Aug 27 02:34:39.626538 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi Aug 27 02:34:39.626538 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi On instance['volume']: Aug 27 02:34:39.626538 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi '#$%' Aug 27 02:34:39.626538 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi Aug 27 02:34:39.626538 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 27 02:34:39.626538 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi Aug 27 02:34:39.626538 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 27 02:34:39.626538 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 27 02:34:39.626538 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 27 02:34:39.626538 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Aug 27 02:34:39.626538 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Aug 27 02:34:39.628146 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Aug 27 02:34:39.628146 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Aug 27 02:34:39.628146 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Aug 27 02:34:39.628146 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Aug 27 02:34:39.628146 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 107, in wrapper Aug 27 02:34:39.628146 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi _schema_validation_helper(request_body_schema, kwargs['body'], Aug 27 02:34:39.628146 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 88, in _schema_validation_helper Aug 27 02:34:39.628146 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi schema_validator.validate(target) Aug 27 02:34:39.628146 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/validators.py", line 331, in validate Aug 27 02:34:39.628146 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi raise exception.ValidationError(detail=detail) Aug 27 02:34:39.628146 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi nova.exception.ValidationError: Invalid input for field/attribute size. Value: #$%. '#$%' does not match '^[0-9]+$' Aug 27 02:34:39.628146 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi Aug 27 02:34:39.628146 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 27 02:34:39.628146 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi Aug 27 02:34:39.628146 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 27 02:34:39.628146 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 27 02:34:39.628146 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 27 02:34:39.628146 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 359, in __exit__ Aug 27 02:34:39.628146 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi raise Fault(exception.ConvertedException( Aug 27 02:34:39.629684 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid input for field/attribute size. Value: #$%. '#$%' does not match '^[0-9]+$' Aug 27 02:34:39.629684 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi Aug 27 02:34:39.629684 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-54293fae-c6c8-427b-ad9d-28d0e3cb1bc4 tempest-VolumesNegativeTest-179747936 tempest-VolumesNegativeTest-179747936-project] Returning 400 to user: Invalid input for field/attribute size. Value: #$%. '#$%' does not match '^[0-9]+$' {{(pid=120864) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Aug 27 02:34:39.629684 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-54293fae-c6c8-427b-ad9d-28d0e3cb1bc4 tempest-VolumesNegativeTest-179747936 tempest-VolumesNegativeTest-179747936-project] 10.222.0.52 "POST /compute/v2.1/os-volumes" status: 400 len: 127 microversion: 2.1 time: 0.139731 Aug 27 02:34:39.629684 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 512/1017] 10.222.0.52 () {60 vars in 1156 bytes} [Sat Aug 27 02:34:39 2022] POST /compute/v2.1/os-volumes => generated 127 bytes in 142 msecs (HTTP/1.1 400) 9 headers in 381 bytes (1 switches on core 0) Aug 27 02:34:39.635442 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-03981445-5a03-4501-9973-c081aa1f9519 tempest-VolumesNegativeTest-179747936 tempest-VolumesNegativeTest-179747936-project] Action: 'create', calling method: .version_select at 0x7f6b1b0f1160>, body: {"volume": {"size": "0", "display_name": "tempest-VolumesNegativeTest-Volume-232966566", "metadata": {"Type": "work"}}} {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 27 02:34:39.640844 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-03981445-5a03-4501-9973-c081aa1f9519 tempest-VolumesNegativeTest-179747936 tempest-VolumesNegativeTest-179747936-project] Request method failure captured: Aug 27 02:34:39.640844 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: request: POST /compute/v2.1/os-volumes HTTP/1.1 Aug 27 02:34:39.640844 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: Accept: application/json Aug 27 02:34:39.640844 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: Accept-Encoding: identity Aug 27 02:34:39.640844 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: Connection: close Aug 27 02:34:39.640844 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: Content-Length: 119 Aug 27 02:34:39.640844 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: Content-Type: application/json Aug 27 02:34:39.640844 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: Host: 10.222.0.52 Aug 27 02:34:39.640844 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: Openstack-System-Scope: None Aug 27 02:34:39.640844 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: User-Agent: python-urllib3/1.26.11 Aug 27 02:34:39.640844 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-Auth-Token: gAAAAABjCYK_pglciF0SMH3Qi_0urNVub6blq0nOW-_hsh0k3tlzIs0UWRxF1356dcQqMh2udDm-HCp0Lfw30RRl9YP-HPrDCF977-mCIcasgDMbn7J6kIPOHllyhaiDe3rMrTupqETInorHI8eCV02vLsldzFsLBOSeSDQ8nVhC03wxGZpOCU0 Aug 27 02:34:39.640844 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-Domain-Id: None Aug 27 02:34:39.640844 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-Domain-Name: None Aug 27 02:34:39.640844 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-Identity-Status: Confirmed Aug 27 02:34:39.640844 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-Is-Admin-Project: True Aug 27 02:34:39.640844 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-Project-Domain-Id: default Aug 27 02:34:39.640844 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-Project-Domain-Name: Default Aug 27 02:34:39.640844 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-Project-Id: 5d046a832eac49fda2b3efd3315cda88 Aug 27 02:34:39.640844 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-Project-Name: tempest-VolumesNegativeTest-179747936 Aug 27 02:34:39.640844 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-Role: reader,member Aug 27 02:34:39.640844 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-Roles: reader,member Aug 27 02:34:39.643447 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-Service-Catalog: [{"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.52:8080", "publicURL": "http://10.222.0.52:8080/v1/AUTH_5d046a832eac49fda2b3efd3315cda88"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52:9696/networking"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/compute/v2/5d046a832eac49fda2b3efd3315cda88"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/image"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/volume/v3/5d046a832eac49fda2b3efd3315cda88"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/identity"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/volume/v3/5d046a832eac49fda2b3efd3315cda88"}]}] Aug 27 02:34:39.643447 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-Tenant: tempest-VolumesNegativeTest-179747936 Aug 27 02:34:39.643447 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-Tenant-Id: 5d046a832eac49fda2b3efd3315cda88 Aug 27 02:34:39.643447 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-Tenant-Name: tempest-VolumesNegativeTest-179747936 Aug 27 02:34:39.643447 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-User: tempest-VolumesNegativeTest-179747936-project Aug 27 02:34:39.643447 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-User-Domain-Id: default Aug 27 02:34:39.643447 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-User-Domain-Name: Default Aug 27 02:34:39.643447 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-User-Id: 03f2912b300d4db392def0d602019c65 Aug 27 02:34:39.643447 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-User-Name: tempest-VolumesNegativeTest-179747936-project Aug 27 02:34:39.643447 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: Aug 27 02:34:39.643447 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: {"volume": {"size": "0", "display_name": "tempest-VolumesNegativeTest-Volume-232966566", "metadata": {"Type": "work"}}} Aug 27 02:34:39.643447 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: method: Aug 27 02:34:39.643447 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: exception: Invalid input for field/attribute size. Value: 0. 0 is less than the minimum of 1 Aug 27 02:34:39.644891 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: action_args: {'body': {'volume': {'size': '0', 'display_name': 'tempest-VolumesNegativeTest-Volume-232966566', 'metadata': {'Type': 'work'}}}} Aug 27 02:34:39.644891 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Aug 27 02:34:39.644891 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 27 02:34:39.644891 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/validators.py", line 302, in validate Aug 27 02:34:39.644891 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi self.validator.validate(*args, **kwargs) Aug 27 02:34:39.644891 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/jsonschema/validators.py", line 268, in validate Aug 27 02:34:39.644891 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi raise error Aug 27 02:34:39.644891 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi jsonschema.exceptions.ValidationError: 0 is less than the minimum of 1 Aug 27 02:34:39.644891 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi Aug 27 02:34:39.644891 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi Failed validating 'minimum' in schema['properties']['volume']['properties']['size']: Aug 27 02:34:39.644891 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi {'minimum': 1, 'pattern': '^[0-9]+$', 'type': ['integer', 'string']} Aug 27 02:34:39.644891 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi Aug 27 02:34:39.644891 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi On instance['volume']: Aug 27 02:34:39.644891 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi '0' Aug 27 02:34:39.644891 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi Aug 27 02:34:39.644891 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 27 02:34:39.644891 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi Aug 27 02:34:39.644891 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 27 02:34:39.644891 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 27 02:34:39.644891 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 27 02:34:39.644891 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Aug 27 02:34:39.644891 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Aug 27 02:34:39.647117 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Aug 27 02:34:39.647117 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Aug 27 02:34:39.647117 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Aug 27 02:34:39.647117 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Aug 27 02:34:39.647117 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 107, in wrapper Aug 27 02:34:39.647117 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi _schema_validation_helper(request_body_schema, kwargs['body'], Aug 27 02:34:39.647117 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 88, in _schema_validation_helper Aug 27 02:34:39.647117 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi schema_validator.validate(target) Aug 27 02:34:39.647117 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/validators.py", line 331, in validate Aug 27 02:34:39.647117 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi raise exception.ValidationError(detail=detail) Aug 27 02:34:39.647117 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi nova.exception.ValidationError: Invalid input for field/attribute size. Value: 0. 0 is less than the minimum of 1 Aug 27 02:34:39.647117 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi Aug 27 02:34:39.647117 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 27 02:34:39.647117 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi Aug 27 02:34:39.647117 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 27 02:34:39.647117 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 27 02:34:39.647117 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 27 02:34:39.647117 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 359, in __exit__ Aug 27 02:34:39.647117 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi raise Fault(exception.ConvertedException( Aug 27 02:34:39.649229 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid input for field/attribute size. Value: 0. 0 is less than the minimum of 1 Aug 27 02:34:39.649229 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi Aug 27 02:34:39.649229 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-03981445-5a03-4501-9973-c081aa1f9519 tempest-VolumesNegativeTest-179747936 tempest-VolumesNegativeTest-179747936-project] Returning 400 to user: Invalid input for field/attribute size. Value: 0. 0 is less than the minimum of 1 {{(pid=120863) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Aug 27 02:34:39.649229 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-03981445-5a03-4501-9973-c081aa1f9519 tempest-VolumesNegativeTest-179747936 tempest-VolumesNegativeTest-179747936-project] 10.222.0.52 "POST /compute/v2.1/os-volumes" status: 400 len: 125 microversion: 2.1 time: 0.010995 Aug 27 02:34:39.649229 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 506/1018] 10.222.0.52 () {60 vars in 1156 bytes} [Sat Aug 27 02:34:39 2022] POST /compute/v2.1/os-volumes => generated 125 bytes in 12 msecs (HTTP/1.1 400) 9 headers in 381 bytes (1 switches on core 0) Aug 27 02:34:39.653071 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-e40a303d-83e4-4465-a105-3a61f180ead9 tempest-VolumesNegativeTest-179747936 tempest-VolumesNegativeTest-179747936-project] Action: 'create', calling method: .version_select at 0x7f6b1e7db670>, body: {"volume": {"size": "", "display_name": "tempest-VolumesNegativeTest-Volume-1750470495", "metadata": {"Type": "work"}}} {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 27 02:34:39.657859 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-e40a303d-83e4-4465-a105-3a61f180ead9 tempest-VolumesNegativeTest-179747936 tempest-VolumesNegativeTest-179747936-project] Request method failure captured: Aug 27 02:34:39.657859 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: request: POST /compute/v2.1/os-volumes HTTP/1.1 Aug 27 02:34:39.657859 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: Accept: application/json Aug 27 02:34:39.657859 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: Accept-Encoding: identity Aug 27 02:34:39.657859 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: Connection: close Aug 27 02:34:39.657859 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: Content-Length: 119 Aug 27 02:34:39.657859 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: Content-Type: application/json Aug 27 02:34:39.657859 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: Host: 10.222.0.52 Aug 27 02:34:39.657859 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: Openstack-System-Scope: None Aug 27 02:34:39.657859 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: User-Agent: python-urllib3/1.26.11 Aug 27 02:34:39.657859 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-Auth-Token: gAAAAABjCYK_pglciF0SMH3Qi_0urNVub6blq0nOW-_hsh0k3tlzIs0UWRxF1356dcQqMh2udDm-HCp0Lfw30RRl9YP-HPrDCF977-mCIcasgDMbn7J6kIPOHllyhaiDe3rMrTupqETInorHI8eCV02vLsldzFsLBOSeSDQ8nVhC03wxGZpOCU0 Aug 27 02:34:39.657859 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-Domain-Id: None Aug 27 02:34:39.657859 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-Domain-Name: None Aug 27 02:34:39.657859 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-Identity-Status: Confirmed Aug 27 02:34:39.657859 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-Is-Admin-Project: True Aug 27 02:34:39.657859 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-Project-Domain-Id: default Aug 27 02:34:39.657859 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-Project-Domain-Name: Default Aug 27 02:34:39.657859 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-Project-Id: 5d046a832eac49fda2b3efd3315cda88 Aug 27 02:34:39.657859 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-Project-Name: tempest-VolumesNegativeTest-179747936 Aug 27 02:34:39.657859 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-Role: reader,member Aug 27 02:34:39.657859 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-Roles: reader,member Aug 27 02:34:39.660285 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-Service-Catalog: [{"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.52:8080", "publicURL": "http://10.222.0.52:8080/v1/AUTH_5d046a832eac49fda2b3efd3315cda88"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52:9696/networking"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/compute/v2/5d046a832eac49fda2b3efd3315cda88"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/image"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/volume/v3/5d046a832eac49fda2b3efd3315cda88"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/identity"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/volume/v3/5d046a832eac49fda2b3efd3315cda88"}]}] Aug 27 02:34:39.660285 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-Tenant: tempest-VolumesNegativeTest-179747936 Aug 27 02:34:39.660285 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-Tenant-Id: 5d046a832eac49fda2b3efd3315cda88 Aug 27 02:34:39.660285 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-Tenant-Name: tempest-VolumesNegativeTest-179747936 Aug 27 02:34:39.660285 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-User: tempest-VolumesNegativeTest-179747936-project Aug 27 02:34:39.660285 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-User-Domain-Id: default Aug 27 02:34:39.660285 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-User-Domain-Name: Default Aug 27 02:34:39.660285 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-User-Id: 03f2912b300d4db392def0d602019c65 Aug 27 02:34:39.660285 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-User-Name: tempest-VolumesNegativeTest-179747936-project Aug 27 02:34:39.660285 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: Aug 27 02:34:39.660285 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: {"volume": {"size": "", "display_name": "tempest-VolumesNegativeTest-Volume-1750470495", "metadata": {"Type": "work"}}} Aug 27 02:34:39.660285 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: method: Aug 27 02:34:39.660285 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: exception: Invalid input for field/attribute size. Value: . '' does not match '^[0-9]+$' Aug 27 02:34:39.661286 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: action_args: {'body': {'volume': {'size': '', 'display_name': 'tempest-VolumesNegativeTest-Volume-1750470495', 'metadata': {'Type': 'work'}}}} Aug 27 02:34:39.661286 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Aug 27 02:34:39.661286 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 27 02:34:39.661286 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/validators.py", line 302, in validate Aug 27 02:34:39.661286 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi self.validator.validate(*args, **kwargs) Aug 27 02:34:39.661286 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/jsonschema/validators.py", line 268, in validate Aug 27 02:34:39.661286 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi raise error Aug 27 02:34:39.661286 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi jsonschema.exceptions.ValidationError: '' does not match '^[0-9]+$' Aug 27 02:34:39.661286 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi Aug 27 02:34:39.661286 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi Failed validating 'pattern' in schema['properties']['volume']['properties']['size']: Aug 27 02:34:39.661286 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi {'minimum': 1, 'pattern': '^[0-9]+$', 'type': ['integer', 'string']} Aug 27 02:34:39.661286 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi Aug 27 02:34:39.661286 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi On instance['volume']: Aug 27 02:34:39.661286 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi '' Aug 27 02:34:39.661286 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi Aug 27 02:34:39.661286 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 27 02:34:39.661286 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi Aug 27 02:34:39.661286 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 27 02:34:39.661286 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 27 02:34:39.661286 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 27 02:34:39.661286 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Aug 27 02:34:39.661286 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Aug 27 02:34:39.663074 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Aug 27 02:34:39.663074 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Aug 27 02:34:39.663074 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Aug 27 02:34:39.663074 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Aug 27 02:34:39.663074 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 107, in wrapper Aug 27 02:34:39.663074 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi _schema_validation_helper(request_body_schema, kwargs['body'], Aug 27 02:34:39.663074 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 88, in _schema_validation_helper Aug 27 02:34:39.663074 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi schema_validator.validate(target) Aug 27 02:34:39.663074 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/validators.py", line 331, in validate Aug 27 02:34:39.663074 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi raise exception.ValidationError(detail=detail) Aug 27 02:34:39.663074 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi nova.exception.ValidationError: Invalid input for field/attribute size. Value: . '' does not match '^[0-9]+$' Aug 27 02:34:39.663074 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi Aug 27 02:34:39.663074 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 27 02:34:39.663074 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi Aug 27 02:34:39.663074 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 27 02:34:39.663074 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 27 02:34:39.663074 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 27 02:34:39.663074 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 359, in __exit__ Aug 27 02:34:39.663074 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi raise Fault(exception.ConvertedException( Aug 27 02:34:39.664450 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid input for field/attribute size. Value: . '' does not match '^[0-9]+$' Aug 27 02:34:39.664450 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi Aug 27 02:34:39.664450 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-e40a303d-83e4-4465-a105-3a61f180ead9 tempest-VolumesNegativeTest-179747936 tempest-VolumesNegativeTest-179747936-project] Returning 400 to user: Invalid input for field/attribute size. Value: . '' does not match '^[0-9]+$' {{(pid=120864) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Aug 27 02:34:39.664450 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-e40a303d-83e4-4465-a105-3a61f180ead9 tempest-VolumesNegativeTest-179747936 tempest-VolumesNegativeTest-179747936-project] 10.222.0.52 "POST /compute/v2.1/os-volumes" status: 400 len: 121 microversion: 2.1 time: 0.010183 Aug 27 02:34:39.664450 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 513/1019] 10.222.0.52 () {60 vars in 1156 bytes} [Sat Aug 27 02:34:39 2022] POST /compute/v2.1/os-volumes => generated 121 bytes in 11 msecs (HTTP/1.1 400) 9 headers in 381 bytes (2 switches on core 0) Aug 27 02:34:39.671446 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-ee08b178-e458-4be5-8b2d-8988412b2dda tempest-VolumesNegativeTest-179747936 tempest-VolumesNegativeTest-179747936-project] Calling method '.version_select at 0x7f6b1b223f70>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:34:39.706304 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.wsgi [None req-ee08b178-e458-4be5-8b2d-8988412b2dda tempest-VolumesNegativeTest-179747936 tempest-VolumesNegativeTest-179747936-project] HTTP exception thrown: Volume tempest-invalid-46469117 could not be found. Aug 27 02:34:39.708301 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-ee08b178-e458-4be5-8b2d-8988412b2dda tempest-VolumesNegativeTest-179747936 tempest-VolumesNegativeTest-179747936-project] Request method failure captured: Aug 27 02:34:39.708301 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: request: DELETE /compute/v2.1/os-volumes/tempest-invalid-46469117 HTTP/1.1 Aug 27 02:34:39.708301 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: Accept: application/json Aug 27 02:34:39.708301 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: Accept-Encoding: identity Aug 27 02:34:39.708301 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: Connection: close Aug 27 02:34:39.708301 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: Content-Length: 0 Aug 27 02:34:39.708301 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: Content-Type: application/json Aug 27 02:34:39.708301 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: Host: 10.222.0.52 Aug 27 02:34:39.708301 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: Openstack-System-Scope: None Aug 27 02:34:39.708301 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: User-Agent: python-urllib3/1.26.11 Aug 27 02:34:39.708301 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-Auth-Token: gAAAAABjCYK_pglciF0SMH3Qi_0urNVub6blq0nOW-_hsh0k3tlzIs0UWRxF1356dcQqMh2udDm-HCp0Lfw30RRl9YP-HPrDCF977-mCIcasgDMbn7J6kIPOHllyhaiDe3rMrTupqETInorHI8eCV02vLsldzFsLBOSeSDQ8nVhC03wxGZpOCU0 Aug 27 02:34:39.708301 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-Domain-Id: None Aug 27 02:34:39.708301 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-Domain-Name: None Aug 27 02:34:39.708301 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-Identity-Status: Confirmed Aug 27 02:34:39.708301 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-Is-Admin-Project: True Aug 27 02:34:39.708301 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-Project-Domain-Id: default Aug 27 02:34:39.708301 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-Project-Domain-Name: Default Aug 27 02:34:39.708301 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-Project-Id: 5d046a832eac49fda2b3efd3315cda88 Aug 27 02:34:39.708301 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-Project-Name: tempest-VolumesNegativeTest-179747936 Aug 27 02:34:39.708301 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-Role: reader,member Aug 27 02:34:39.708301 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-Roles: reader,member Aug 27 02:34:39.710028 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-Service-Catalog: [{"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.52:8080", "publicURL": "http://10.222.0.52:8080/v1/AUTH_5d046a832eac49fda2b3efd3315cda88"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52:9696/networking"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/compute/v2/5d046a832eac49fda2b3efd3315cda88"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/image"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/volume/v3/5d046a832eac49fda2b3efd3315cda88"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/identity"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/volume/v3/5d046a832eac49fda2b3efd3315cda88"}]}] Aug 27 02:34:39.710028 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-Tenant: tempest-VolumesNegativeTest-179747936 Aug 27 02:34:39.710028 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-Tenant-Id: 5d046a832eac49fda2b3efd3315cda88 Aug 27 02:34:39.710028 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-Tenant-Name: tempest-VolumesNegativeTest-179747936 Aug 27 02:34:39.710028 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-User: tempest-VolumesNegativeTest-179747936-project Aug 27 02:34:39.710028 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-User-Domain-Id: default Aug 27 02:34:39.710028 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-User-Domain-Name: Default Aug 27 02:34:39.710028 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-User-Id: 03f2912b300d4db392def0d602019c65 Aug 27 02:34:39.710028 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-User-Name: tempest-VolumesNegativeTest-179747936-project Aug 27 02:34:39.710028 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: method: Aug 27 02:34:39.710028 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: exception: Volume tempest-invalid-46469117 could not be found. Aug 27 02:34:39.710028 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: action_args: {'id': 'tempest-invalid-46469117'} Aug 27 02:34:39.710028 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Aug 27 02:34:39.716320 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 27 02:34:39.716320 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Aug 27 02:34:39.716320 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Aug 27 02:34:39.716320 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 676, in delete Aug 27 02:34:39.716320 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.delete(volume_id) Aug 27 02:34:39.716320 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 235, in delete Aug 27 02:34:39.716320 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi return self._delete(loc) Aug 27 02:34:39.716320 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 313, in _delete Aug 27 02:34:39.716320 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.delete(url) Aug 27 02:34:39.716320 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 233, in delete Aug 27 02:34:39.716320 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'DELETE', **kwargs) Aug 27 02:34:39.716320 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Aug 27 02:34:39.716320 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Aug 27 02:34:39.716320 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Aug 27 02:34:39.716320 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Aug 27 02:34:39.716320 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.NotFound: Volume tempest-invalid-46469117 could not be found. (HTTP 404) (Request-ID: req-008cadf9-1e91-4e92-830a-f10a1d19fd4c) Aug 27 02:34:39.716320 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi Aug 27 02:34:39.716320 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 27 02:34:39.716320 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi Aug 27 02:34:39.717950 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 27 02:34:39.717950 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 136, in delete Aug 27 02:34:39.717950 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi self.volume_api.delete(context, id) Aug 27 02:34:39.717950 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Aug 27 02:34:39.717950 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Aug 27 02:34:39.717950 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 435, in wrapper Aug 27 02:34:39.717950 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi _reraise(exception.VolumeNotFound(volume_id=volume_id)) Aug 27 02:34:39.717950 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise Aug 27 02:34:39.717950 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) Aug 27 02:34:39.717950 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Aug 27 02:34:39.717950 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Aug 27 02:34:39.717950 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 676, in delete Aug 27 02:34:39.717950 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.delete(volume_id) Aug 27 02:34:39.717950 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 235, in delete Aug 27 02:34:39.717950 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi return self._delete(loc) Aug 27 02:34:39.717950 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 313, in _delete Aug 27 02:34:39.717950 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.delete(url) Aug 27 02:34:39.717950 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 233, in delete Aug 27 02:34:39.717950 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'DELETE', **kwargs) Aug 27 02:34:39.719399 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Aug 27 02:34:39.719399 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Aug 27 02:34:39.719399 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Aug 27 02:34:39.719399 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Aug 27 02:34:39.719399 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi nova.exception.VolumeNotFound: Volume tempest-invalid-46469117 could not be found. Aug 27 02:34:39.719399 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi Aug 27 02:34:39.719399 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 27 02:34:39.719399 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi Aug 27 02:34:39.719399 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 27 02:34:39.719399 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 27 02:34:39.719399 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 27 02:34:39.719399 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Aug 27 02:34:39.719399 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Aug 27 02:34:39.719399 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Aug 27 02:34:39.719399 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Aug 27 02:34:39.719399 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Aug 27 02:34:39.719399 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Aug 27 02:34:39.719399 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 140, in delete Aug 27 02:34:39.719399 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Aug 27 02:34:39.719399 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Volume tempest-invalid-46469117 could not be found. Aug 27 02:34:39.721514 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi Aug 27 02:34:39.721514 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 27 02:34:39.721514 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi Aug 27 02:34:39.721514 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 27 02:34:39.721514 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 27 02:34:39.721514 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 27 02:34:39.721514 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Aug 27 02:34:39.721514 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Aug 27 02:34:39.721514 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Volume tempest-invalid-46469117 could not be found. Aug 27 02:34:39.721514 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi Aug 27 02:34:39.721514 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-ee08b178-e458-4be5-8b2d-8988412b2dda tempest-VolumesNegativeTest-179747936 tempest-VolumesNegativeTest-179747936-project] Returning 404 to user: Volume tempest-invalid-46469117 could not be found. {{(pid=120863) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Aug 27 02:34:39.721514 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-ee08b178-e458-4be5-8b2d-8988412b2dda tempest-VolumesNegativeTest-179747936 tempest-VolumesNegativeTest-179747936-project] 10.222.0.52 "DELETE /compute/v2.1/os-volumes/tempest-invalid-46469117" status: 404 len: 97 microversion: 2.1 time: 0.043319 Aug 27 02:34:39.721514 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 507/1020] 10.222.0.52 () {58 vars in 1212 bytes} [Sat Aug 27 02:34:39 2022] DELETE /compute/v2.1/os-volumes/tempest-invalid-46469117 => generated 97 bytes in 45 msecs (HTTP/1.1 404) 9 headers in 378 bytes (1 switches on core 0) Aug 27 02:34:39.726586 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-653f07b1-f485-4e24-9116-f5da1219fb0f tempest-VolumesNegativeTest-179747936 tempest-VolumesNegativeTest-179747936-project] 10.222.0.52 "DELETE /compute/v2.1/os-volumes/" status: 404 len: 112 microversion: - time: 0.006646 Aug 27 02:34:39.727135 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 514/1021] 10.222.0.52 () {58 vars in 1140 bytes} [Sat Aug 27 02:34:39 2022] DELETE /compute/v2.1/os-volumes/ => generated 112 bytes in 11 msecs (HTTP/1.1 404) 5 headers in 228 bytes (1 switches on core 0) Aug 27 02:34:39.738224 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-f18b4d64-4a86-4616-997e-79c40f4e968f tempest-VolumesNegativeTest-179747936 tempest-VolumesNegativeTest-179747936-project] 10.222.0.52 "GET /compute/v2.1/os-volumes/" status: 404 len: 112 microversion: - time: 0.003899 Aug 27 02:34:39.738652 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 508/1022] 10.222.0.52 () {58 vars in 1137 bytes} [Sat Aug 27 02:34:39 2022] GET /compute/v2.1/os-volumes/ => generated 112 bytes in 6 msecs (HTTP/1.1 404) 5 headers in 228 bytes (2 switches on core 0) Aug 27 02:34:39.747970 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-4b37fdf7-78cb-4c69-bffa-6e18902edcd4 tempest-VolumesNegativeTest-179747936 tempest-VolumesNegativeTest-179747936-project] Calling method '.version_select at 0x7f6b1e8928b0>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:34:39.780698 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.wsgi [None req-4b37fdf7-78cb-4c69-bffa-6e18902edcd4 tempest-VolumesNegativeTest-179747936 tempest-VolumesNegativeTest-179747936-project] HTTP exception thrown: Volume 041eb6ea-4471-468e-80b6-1a3c8d758d97 could not be found. Aug 27 02:34:39.782663 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-4b37fdf7-78cb-4c69-bffa-6e18902edcd4 tempest-VolumesNegativeTest-179747936 tempest-VolumesNegativeTest-179747936-project] Request method failure captured: Aug 27 02:34:39.782663 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: request: DELETE /compute/v2.1/os-volumes/041eb6ea-4471-468e-80b6-1a3c8d758d97 HTTP/1.1 Aug 27 02:34:39.782663 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: Accept: application/json Aug 27 02:34:39.782663 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: Accept-Encoding: identity Aug 27 02:34:39.782663 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: Connection: close Aug 27 02:34:39.782663 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: Content-Length: 0 Aug 27 02:34:39.782663 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: Content-Type: application/json Aug 27 02:34:39.782663 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: Host: 10.222.0.52 Aug 27 02:34:39.782663 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: Openstack-System-Scope: None Aug 27 02:34:39.782663 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: User-Agent: python-urllib3/1.26.11 Aug 27 02:34:39.782663 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-Auth-Token: gAAAAABjCYK_pglciF0SMH3Qi_0urNVub6blq0nOW-_hsh0k3tlzIs0UWRxF1356dcQqMh2udDm-HCp0Lfw30RRl9YP-HPrDCF977-mCIcasgDMbn7J6kIPOHllyhaiDe3rMrTupqETInorHI8eCV02vLsldzFsLBOSeSDQ8nVhC03wxGZpOCU0 Aug 27 02:34:39.782663 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-Domain-Id: None Aug 27 02:34:39.782663 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-Domain-Name: None Aug 27 02:34:39.782663 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-Identity-Status: Confirmed Aug 27 02:34:39.782663 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-Is-Admin-Project: True Aug 27 02:34:39.782663 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-Project-Domain-Id: default Aug 27 02:34:39.782663 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-Project-Domain-Name: Default Aug 27 02:34:39.782663 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-Project-Id: 5d046a832eac49fda2b3efd3315cda88 Aug 27 02:34:39.782663 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-Project-Name: tempest-VolumesNegativeTest-179747936 Aug 27 02:34:39.782663 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-Role: reader,member Aug 27 02:34:39.782663 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-Roles: reader,member Aug 27 02:34:39.784519 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-Service-Catalog: [{"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.52:8080", "publicURL": "http://10.222.0.52:8080/v1/AUTH_5d046a832eac49fda2b3efd3315cda88"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52:9696/networking"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/compute/v2/5d046a832eac49fda2b3efd3315cda88"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/image"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/volume/v3/5d046a832eac49fda2b3efd3315cda88"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/identity"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/volume/v3/5d046a832eac49fda2b3efd3315cda88"}]}] Aug 27 02:34:39.784519 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-Tenant: tempest-VolumesNegativeTest-179747936 Aug 27 02:34:39.784519 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-Tenant-Id: 5d046a832eac49fda2b3efd3315cda88 Aug 27 02:34:39.784519 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-Tenant-Name: tempest-VolumesNegativeTest-179747936 Aug 27 02:34:39.784519 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-User: tempest-VolumesNegativeTest-179747936-project Aug 27 02:34:39.784519 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-User-Domain-Id: default Aug 27 02:34:39.784519 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-User-Domain-Name: Default Aug 27 02:34:39.784519 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-User-Id: 03f2912b300d4db392def0d602019c65 Aug 27 02:34:39.784519 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-User-Name: tempest-VolumesNegativeTest-179747936-project Aug 27 02:34:39.784519 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: method: Aug 27 02:34:39.784519 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: exception: Volume 041eb6ea-4471-468e-80b6-1a3c8d758d97 could not be found. Aug 27 02:34:39.784519 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: action_args: {'id': '041eb6ea-4471-468e-80b6-1a3c8d758d97'} Aug 27 02:34:39.784519 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Aug 27 02:34:39.785521 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 27 02:34:39.785521 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Aug 27 02:34:39.785521 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Aug 27 02:34:39.785521 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 676, in delete Aug 27 02:34:39.785521 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.delete(volume_id) Aug 27 02:34:39.785521 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 235, in delete Aug 27 02:34:39.785521 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi return self._delete(loc) Aug 27 02:34:39.785521 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 313, in _delete Aug 27 02:34:39.785521 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.delete(url) Aug 27 02:34:39.785521 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 233, in delete Aug 27 02:34:39.785521 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'DELETE', **kwargs) Aug 27 02:34:39.785521 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Aug 27 02:34:39.785521 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Aug 27 02:34:39.785521 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Aug 27 02:34:39.785521 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Aug 27 02:34:39.785521 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.NotFound: Volume 041eb6ea-4471-468e-80b6-1a3c8d758d97 could not be found. (HTTP 404) (Request-ID: req-0216eb4d-ea1a-42c9-86c4-dd3b1be0a0d0) Aug 27 02:34:39.785521 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi Aug 27 02:34:39.785521 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 27 02:34:39.785521 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi Aug 27 02:34:39.786877 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 27 02:34:39.786877 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 136, in delete Aug 27 02:34:39.786877 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi self.volume_api.delete(context, id) Aug 27 02:34:39.786877 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Aug 27 02:34:39.786877 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Aug 27 02:34:39.786877 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 435, in wrapper Aug 27 02:34:39.786877 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi _reraise(exception.VolumeNotFound(volume_id=volume_id)) Aug 27 02:34:39.786877 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise Aug 27 02:34:39.786877 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) Aug 27 02:34:39.786877 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Aug 27 02:34:39.786877 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Aug 27 02:34:39.786877 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 676, in delete Aug 27 02:34:39.786877 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.delete(volume_id) Aug 27 02:34:39.786877 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 235, in delete Aug 27 02:34:39.786877 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi return self._delete(loc) Aug 27 02:34:39.786877 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 313, in _delete Aug 27 02:34:39.786877 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.delete(url) Aug 27 02:34:39.786877 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 233, in delete Aug 27 02:34:39.786877 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'DELETE', **kwargs) Aug 27 02:34:39.788346 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Aug 27 02:34:39.788346 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Aug 27 02:34:39.788346 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Aug 27 02:34:39.788346 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Aug 27 02:34:39.788346 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi nova.exception.VolumeNotFound: Volume 041eb6ea-4471-468e-80b6-1a3c8d758d97 could not be found. Aug 27 02:34:39.788346 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi Aug 27 02:34:39.788346 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 27 02:34:39.788346 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi Aug 27 02:34:39.788346 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 27 02:34:39.788346 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 27 02:34:39.788346 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 27 02:34:39.788346 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Aug 27 02:34:39.788346 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Aug 27 02:34:39.788346 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Aug 27 02:34:39.788346 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Aug 27 02:34:39.788346 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Aug 27 02:34:39.788346 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Aug 27 02:34:39.788346 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 140, in delete Aug 27 02:34:39.788346 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Aug 27 02:34:39.789761 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Volume 041eb6ea-4471-468e-80b6-1a3c8d758d97 could not be found. Aug 27 02:34:39.789761 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi Aug 27 02:34:39.789761 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 27 02:34:39.789761 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi Aug 27 02:34:39.789761 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 27 02:34:39.789761 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 27 02:34:39.789761 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 27 02:34:39.789761 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Aug 27 02:34:39.789761 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Aug 27 02:34:39.789761 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Volume 041eb6ea-4471-468e-80b6-1a3c8d758d97 could not be found. Aug 27 02:34:39.789761 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi Aug 27 02:34:39.789761 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-4b37fdf7-78cb-4c69-bffa-6e18902edcd4 tempest-VolumesNegativeTest-179747936 tempest-VolumesNegativeTest-179747936-project] Returning 404 to user: Volume 041eb6ea-4471-468e-80b6-1a3c8d758d97 could not be found. {{(pid=120864) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Aug 27 02:34:39.789761 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-4b37fdf7-78cb-4c69-bffa-6e18902edcd4 tempest-VolumesNegativeTest-179747936 tempest-VolumesNegativeTest-179747936-project] 10.222.0.52 "DELETE /compute/v2.1/os-volumes/041eb6ea-4471-468e-80b6-1a3c8d758d97" status: 404 len: 109 microversion: 2.1 time: 0.040023 Aug 27 02:34:39.789761 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 515/1023] 10.222.0.52 () {58 vars in 1248 bytes} [Sat Aug 27 02:34:39 2022] DELETE /compute/v2.1/os-volumes/041eb6ea-4471-468e-80b6-1a3c8d758d97 => generated 109 bytes in 41 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Aug 27 02:34:39.798761 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-baa905e6-1645-42e7-85b2-b7b464b682a7 tempest-VolumesNegativeTest-179747936 tempest-VolumesNegativeTest-179747936-project] Calling method '.version_select at 0x7f6b1b1e1670>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:34:39.832578 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.wsgi [None req-baa905e6-1645-42e7-85b2-b7b464b682a7 tempest-VolumesNegativeTest-179747936 tempest-VolumesNegativeTest-179747936-project] HTTP exception thrown: Volume eeea6c36-1910-4b3b-bbb5-b952e7f572fb could not be found. Aug 27 02:34:39.834909 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-baa905e6-1645-42e7-85b2-b7b464b682a7 tempest-VolumesNegativeTest-179747936 tempest-VolumesNegativeTest-179747936-project] Request method failure captured: Aug 27 02:34:39.834909 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: request: GET /compute/v2.1/os-volumes/eeea6c36-1910-4b3b-bbb5-b952e7f572fb HTTP/1.1 Aug 27 02:34:39.834909 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: Accept: application/json Aug 27 02:34:39.834909 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: Accept-Encoding: identity Aug 27 02:34:39.834909 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: Connection: close Aug 27 02:34:39.834909 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: Content-Length: 0 Aug 27 02:34:39.834909 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: Content-Type: application/json Aug 27 02:34:39.834909 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: Host: 10.222.0.52 Aug 27 02:34:39.834909 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: Openstack-System-Scope: None Aug 27 02:34:39.834909 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: User-Agent: python-urllib3/1.26.11 Aug 27 02:34:39.834909 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-Auth-Token: gAAAAABjCYK_pglciF0SMH3Qi_0urNVub6blq0nOW-_hsh0k3tlzIs0UWRxF1356dcQqMh2udDm-HCp0Lfw30RRl9YP-HPrDCF977-mCIcasgDMbn7J6kIPOHllyhaiDe3rMrTupqETInorHI8eCV02vLsldzFsLBOSeSDQ8nVhC03wxGZpOCU0 Aug 27 02:34:39.834909 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-Domain-Id: None Aug 27 02:34:39.834909 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-Domain-Name: None Aug 27 02:34:39.834909 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-Identity-Status: Confirmed Aug 27 02:34:39.834909 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-Is-Admin-Project: True Aug 27 02:34:39.834909 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-Project-Domain-Id: default Aug 27 02:34:39.834909 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-Project-Domain-Name: Default Aug 27 02:34:39.834909 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-Project-Id: 5d046a832eac49fda2b3efd3315cda88 Aug 27 02:34:39.834909 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-Project-Name: tempest-VolumesNegativeTest-179747936 Aug 27 02:34:39.834909 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-Role: reader,member Aug 27 02:34:39.834909 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-Roles: reader,member Aug 27 02:34:39.836480 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-Service-Catalog: [{"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.52:8080", "publicURL": "http://10.222.0.52:8080/v1/AUTH_5d046a832eac49fda2b3efd3315cda88"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52:9696/networking"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/compute/v2/5d046a832eac49fda2b3efd3315cda88"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/image"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/volume/v3/5d046a832eac49fda2b3efd3315cda88"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/identity"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/volume/v3/5d046a832eac49fda2b3efd3315cda88"}]}] Aug 27 02:34:39.836480 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-Tenant: tempest-VolumesNegativeTest-179747936 Aug 27 02:34:39.836480 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-Tenant-Id: 5d046a832eac49fda2b3efd3315cda88 Aug 27 02:34:39.836480 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-Tenant-Name: tempest-VolumesNegativeTest-179747936 Aug 27 02:34:39.836480 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-User: tempest-VolumesNegativeTest-179747936-project Aug 27 02:34:39.836480 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-User-Domain-Id: default Aug 27 02:34:39.836480 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-User-Domain-Name: Default Aug 27 02:34:39.836480 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-User-Id: 03f2912b300d4db392def0d602019c65 Aug 27 02:34:39.836480 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-User-Name: tempest-VolumesNegativeTest-179747936-project Aug 27 02:34:39.836480 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: method: Aug 27 02:34:39.836480 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: exception: Volume eeea6c36-1910-4b3b-bbb5-b952e7f572fb could not be found. Aug 27 02:34:39.836480 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: action_args: {'id': 'eeea6c36-1910-4b3b-bbb5-b952e7f572fb'} Aug 27 02:34:39.836480 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Aug 27 02:34:39.837464 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 27 02:34:39.837464 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Aug 27 02:34:39.837464 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Aug 27 02:34:39.837464 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 502, in get Aug 27 02:34:39.837464 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi item = cinderclient( Aug 27 02:34:39.837464 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 202, in get Aug 27 02:34:39.837464 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi return self._get("/volumes/%s" % volume_id, "volume") Aug 27 02:34:39.837464 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 297, in _get Aug 27 02:34:39.837464 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.get(url) Aug 27 02:34:39.837464 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 224, in get Aug 27 02:34:39.837464 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'GET', **kwargs) Aug 27 02:34:39.837464 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Aug 27 02:34:39.837464 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Aug 27 02:34:39.837464 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Aug 27 02:34:39.837464 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Aug 27 02:34:39.837464 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.NotFound: Volume eeea6c36-1910-4b3b-bbb5-b952e7f572fb could not be found. (HTTP 404) (Request-ID: req-3c9f524d-c6e0-4ab2-b2e8-97f6ac179ad3) Aug 27 02:34:39.837464 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi Aug 27 02:34:39.837464 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 27 02:34:39.837464 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi Aug 27 02:34:39.838873 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 27 02:34:39.838873 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 120, in show Aug 27 02:34:39.838873 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi vol = self.volume_api.get(context, id) Aug 27 02:34:39.838873 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Aug 27 02:34:39.838873 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Aug 27 02:34:39.838873 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 435, in wrapper Aug 27 02:34:39.838873 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi _reraise(exception.VolumeNotFound(volume_id=volume_id)) Aug 27 02:34:39.838873 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise Aug 27 02:34:39.838873 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) Aug 27 02:34:39.838873 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Aug 27 02:34:39.838873 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Aug 27 02:34:39.838873 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 502, in get Aug 27 02:34:39.838873 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi item = cinderclient( Aug 27 02:34:39.838873 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 202, in get Aug 27 02:34:39.838873 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi return self._get("/volumes/%s" % volume_id, "volume") Aug 27 02:34:39.838873 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 297, in _get Aug 27 02:34:39.838873 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.get(url) Aug 27 02:34:39.838873 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 224, in get Aug 27 02:34:39.838873 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'GET', **kwargs) Aug 27 02:34:39.840255 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Aug 27 02:34:39.840255 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Aug 27 02:34:39.840255 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Aug 27 02:34:39.840255 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Aug 27 02:34:39.840255 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi nova.exception.VolumeNotFound: Volume eeea6c36-1910-4b3b-bbb5-b952e7f572fb could not be found. Aug 27 02:34:39.840255 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi Aug 27 02:34:39.840255 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 27 02:34:39.840255 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi Aug 27 02:34:39.840255 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 27 02:34:39.840255 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 27 02:34:39.840255 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 27 02:34:39.840255 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Aug 27 02:34:39.840255 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Aug 27 02:34:39.840255 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Aug 27 02:34:39.840255 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Aug 27 02:34:39.840255 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Aug 27 02:34:39.840255 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Aug 27 02:34:39.840255 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 122, in show Aug 27 02:34:39.840255 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Aug 27 02:34:39.841734 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Volume eeea6c36-1910-4b3b-bbb5-b952e7f572fb could not be found. Aug 27 02:34:39.841734 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi Aug 27 02:34:39.841734 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 27 02:34:39.841734 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi Aug 27 02:34:39.841734 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 27 02:34:39.841734 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 27 02:34:39.841734 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 27 02:34:39.841734 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Aug 27 02:34:39.841734 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Aug 27 02:34:39.841734 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Volume eeea6c36-1910-4b3b-bbb5-b952e7f572fb could not be found. Aug 27 02:34:39.841734 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi Aug 27 02:34:39.841734 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-baa905e6-1645-42e7-85b2-b7b464b682a7 tempest-VolumesNegativeTest-179747936 tempest-VolumesNegativeTest-179747936-project] Returning 404 to user: Volume eeea6c36-1910-4b3b-bbb5-b952e7f572fb could not be found. {{(pid=120863) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Aug 27 02:34:39.841734 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-baa905e6-1645-42e7-85b2-b7b464b682a7 tempest-VolumesNegativeTest-179747936 tempest-VolumesNegativeTest-179747936-project] 10.222.0.52 "GET /compute/v2.1/os-volumes/eeea6c36-1910-4b3b-bbb5-b952e7f572fb" status: 404 len: 109 microversion: 2.1 time: 0.045164 Aug 27 02:34:39.841734 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 509/1024] 10.222.0.52 () {58 vars in 1245 bytes} [Sat Aug 27 02:34:39 2022] GET /compute/v2.1/os-volumes/eeea6c36-1910-4b3b-bbb5-b952e7f572fb => generated 109 bytes in 47 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Aug 27 02:35:06.262285 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG dbcounter [-] [120864] Writing DB stats nova_api:SELECT=13 {{(pid=120864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 27 02:35:06.263338 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG dbcounter [-] [120864] Writing DB stats nova_cell1:SELECT=31,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=120864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 27 02:35:06.269641 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-ab3ba026-9963-46d9-89f5-3be3b334a2d2 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Calling method '>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:35:06.271961 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-ab3ba026-9963-46d9-89f5-3be3b334a2d2 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] [instance: fff5c233-ad7f-4022-8582-e8620f32ebd5] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:35:06.279608 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-ab3ba026-9963-46d9-89f5-3be3b334a2d2 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:35:06.280096 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-ab3ba026-9963-46d9-89f5-3be3b334a2d2 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:35:06.280565 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-ab3ba026-9963-46d9-89f5-3be3b334a2d2 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:35:06.281451 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-ab3ba026-9963-46d9-89f5-3be3b334a2d2 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:35:06.282096 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-ab3ba026-9963-46d9-89f5-3be3b334a2d2 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:35:06.282516 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-ab3ba026-9963-46d9-89f5-3be3b334a2d2 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:35:06.316000 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-ab3ba026-9963-46d9-89f5-3be3b334a2d2 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] 10.222.0.52 "GET /compute/v2.1/servers/fff5c233-ad7f-4022-8582-e8620f32ebd5/os-volume_attachments" status: 200 len: 25 microversion: 2.20 time: 0.055070 Aug 27 02:35:06.316581 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 516/1025] 10.222.0.52 () {60 vars in 1343 bytes} [Sat Aug 27 02:35:06 2022] GET /compute/v2.1/servers/fff5c233-ad7f-4022-8582-e8620f32ebd5/os-volume_attachments => generated 25 bytes in 56 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:35:06.382583 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG dbcounter [-] [120863] Writing DB stats nova_api:SELECT=22,nova_api:DELETE=1 {{(pid=120863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 27 02:35:06.383647 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG dbcounter [-] [120863] Writing DB stats nova_cell1:SELECT=42,nova_cell1:INSERT=1 {{(pid=120863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 27 02:35:06.390106 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-3b783606-a234-426b-9e8e-3376367b49ff tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:35:06.390724 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-3b783606-a234-426b-9e8e-3376367b49ff tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] [instance: fff5c233-ad7f-4022-8582-e8620f32ebd5] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:35:06.398924 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-3b783606-a234-426b-9e8e-3376367b49ff tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:35:06.399553 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-3b783606-a234-426b-9e8e-3376367b49ff tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:35:06.400036 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-3b783606-a234-426b-9e8e-3376367b49ff tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:35:06.400857 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-3b783606-a234-426b-9e8e-3376367b49ff tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:35:06.401395 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-3b783606-a234-426b-9e8e-3376367b49ff tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:35:06.401882 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-3b783606-a234-426b-9e8e-3376367b49ff tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:35:06.430324 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-3b783606-a234-426b-9e8e-3376367b49ff tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] [instance: fff5c233-ad7f-4022-8582-e8620f32ebd5] Going to try to terminate instance {{(pid=120863) delete /opt/stack/nova/nova/compute/api.py:2719}} Aug 27 02:35:06.524288 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-3b783606-a234-426b-9e8e-3376367b49ff tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] 10.222.0.52 "DELETE /compute/v2.1/servers/fff5c233-ad7f-4022-8582-e8620f32ebd5" status: 204 len: 0 microversion: 2.20 time: 0.143167 Aug 27 02:35:06.525157 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 510/1026] 10.222.0.52 () {60 vars in 1280 bytes} [Sat Aug 27 02:35:06 2022] DELETE /compute/v2.1/servers/fff5c233-ad7f-4022-8582-e8620f32ebd5 => generated 0 bytes in 145 msecs (HTTP/1.1 204) 9 headers in 365 bytes (1 switches on core 0) Aug 27 02:35:06.534511 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-291ef08f-5c2a-4d13-b1f5-39aae1443332 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Calling method '>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:35:06.535374 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-291ef08f-5c2a-4d13-b1f5-39aae1443332 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] [instance: fff5c233-ad7f-4022-8582-e8620f32ebd5] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:35:06.542371 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-291ef08f-5c2a-4d13-b1f5-39aae1443332 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:35:06.542860 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-291ef08f-5c2a-4d13-b1f5-39aae1443332 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:35:06.543255 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-291ef08f-5c2a-4d13-b1f5-39aae1443332 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:35:06.544215 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-291ef08f-5c2a-4d13-b1f5-39aae1443332 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:35:06.544659 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-291ef08f-5c2a-4d13-b1f5-39aae1443332 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:35:06.545079 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-291ef08f-5c2a-4d13-b1f5-39aae1443332 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:35:06.686246 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-291ef08f-5c2a-4d13-b1f5-39aae1443332 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:35:06.695215 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-291ef08f-5c2a-4d13-b1f5-39aae1443332 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:35:06.696671 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-291ef08f-5c2a-4d13-b1f5-39aae1443332 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:35:06.698083 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-291ef08f-5c2a-4d13-b1f5-39aae1443332 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] 10.222.0.52 "GET /compute/v2.1/servers/fff5c233-ad7f-4022-8582-e8620f32ebd5" status: 200 len: 1725 microversion: 2.20 time: 0.167731 Aug 27 02:35:06.698796 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 517/1027] 10.222.0.52 () {60 vars in 1277 bytes} [Sat Aug 27 02:35:06 2022] GET /compute/v2.1/servers/fff5c233-ad7f-4022-8582-e8620f32ebd5 => generated 1725 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 27 02:35:07.723338 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-c7b0ba33-5fe8-4d46-b56a-517ef03dd3bd tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:35:07.724768 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-c7b0ba33-5fe8-4d46-b56a-517ef03dd3bd tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] [instance: fff5c233-ad7f-4022-8582-e8620f32ebd5] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:35:07.732466 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-c7b0ba33-5fe8-4d46-b56a-517ef03dd3bd tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:35:07.733002 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-c7b0ba33-5fe8-4d46-b56a-517ef03dd3bd tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:35:07.733588 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-c7b0ba33-5fe8-4d46-b56a-517ef03dd3bd tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:35:07.734326 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-c7b0ba33-5fe8-4d46-b56a-517ef03dd3bd tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:35:07.734760 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-c7b0ba33-5fe8-4d46-b56a-517ef03dd3bd tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:35:07.735186 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-c7b0ba33-5fe8-4d46-b56a-517ef03dd3bd tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:35:07.880469 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-c7b0ba33-5fe8-4d46-b56a-517ef03dd3bd tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:35:07.890511 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-c7b0ba33-5fe8-4d46-b56a-517ef03dd3bd tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:35:07.891821 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-c7b0ba33-5fe8-4d46-b56a-517ef03dd3bd tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:35:07.892981 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-c7b0ba33-5fe8-4d46-b56a-517ef03dd3bd tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] 10.222.0.52 "GET /compute/v2.1/servers/fff5c233-ad7f-4022-8582-e8620f32ebd5" status: 200 len: 1725 microversion: 2.20 time: 0.174792 Aug 27 02:35:07.894001 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 511/1028] 10.222.0.52 () {60 vars in 1277 bytes} [Sat Aug 27 02:35:07 2022] GET /compute/v2.1/servers/fff5c233-ad7f-4022-8582-e8620f32ebd5 => generated 1725 bytes in 177 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 27 02:35:08.917416 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-210ebd0d-9dca-4b10-b580-dc96c8dd93d3 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Calling method '>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:35:08.918165 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-210ebd0d-9dca-4b10-b580-dc96c8dd93d3 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] [instance: fff5c233-ad7f-4022-8582-e8620f32ebd5] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:35:08.929009 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-210ebd0d-9dca-4b10-b580-dc96c8dd93d3 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:35:08.929009 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-210ebd0d-9dca-4b10-b580-dc96c8dd93d3 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:35:08.929397 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-210ebd0d-9dca-4b10-b580-dc96c8dd93d3 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:35:08.930169 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-210ebd0d-9dca-4b10-b580-dc96c8dd93d3 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:35:08.930679 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-210ebd0d-9dca-4b10-b580-dc96c8dd93d3 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:35:08.931315 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-210ebd0d-9dca-4b10-b580-dc96c8dd93d3 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:35:09.155375 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-210ebd0d-9dca-4b10-b580-dc96c8dd93d3 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:35:09.167833 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-210ebd0d-9dca-4b10-b580-dc96c8dd93d3 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:35:09.169436 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-210ebd0d-9dca-4b10-b580-dc96c8dd93d3 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:35:09.170620 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-210ebd0d-9dca-4b10-b580-dc96c8dd93d3 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] 10.222.0.52 "GET /compute/v2.1/servers/fff5c233-ad7f-4022-8582-e8620f32ebd5" status: 200 len: 1725 microversion: 2.20 time: 0.257975 Aug 27 02:35:09.171160 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 518/1029] 10.222.0.52 () {60 vars in 1277 bytes} [Sat Aug 27 02:35:08 2022] GET /compute/v2.1/servers/fff5c233-ad7f-4022-8582-e8620f32ebd5 => generated 1725 bytes in 259 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Aug 27 02:35:09.866560 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [req-5e81bd19-99e7-40c5-838e-1c37262eec8a req-04e242b7-1b01-4ca6-8ceb-e8a717f73ff0 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "fff5c233-ad7f-4022-8582-e8620f32ebd5", "name": "network-vif-deleted", "tag": "67fc5822-1682-4324-ae1c-b98d8d2d09e8"}]} {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 27 02:35:09.879017 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [req-5e81bd19-99e7-40c5-838e-1c37262eec8a req-04e242b7-1b01-4ca6-8ceb-e8a717f73ff0 service nova] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:35:09.879878 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [req-5e81bd19-99e7-40c5-838e-1c37262eec8a req-04e242b7-1b01-4ca6-8ceb-e8a717f73ff0 service nova] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:35:09.881257 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [req-5e81bd19-99e7-40c5-838e-1c37262eec8a req-04e242b7-1b01-4ca6-8ceb-e8a717f73ff0 service nova] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:35:09.895299 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.compute.server_external_events [req-5e81bd19-99e7-40c5-838e-1c37262eec8a req-04e242b7-1b01-4ca6-8ceb-e8a717f73ff0 service nova] Creating event network-vif-deleted:67fc5822-1682-4324-ae1c-b98d8d2d09e8 for instance fff5c233-ad7f-4022-8582-e8620f32ebd5 on hv-cinder-84630 Aug 27 02:35:09.903154 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [req-5e81bd19-99e7-40c5-838e-1c37262eec8a req-04e242b7-1b01-4ca6-8ceb-e8a717f73ff0 service nova] 10.222.0.52 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.292432 Aug 27 02:35:09.904434 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 512/1030] 10.222.0.52 () {64 vars in 1317 bytes} [Sat Aug 27 02:35:09 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 294 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 27 02:35:10.198700 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-315281c9-076e-43c0-b9cc-66233f817f48 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Calling method '>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:35:10.200177 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-315281c9-076e-43c0-b9cc-66233f817f48 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] [instance: fff5c233-ad7f-4022-8582-e8620f32ebd5] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:35:10.210338 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-315281c9-076e-43c0-b9cc-66233f817f48 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:35:10.211157 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-315281c9-076e-43c0-b9cc-66233f817f48 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:35:10.211831 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-315281c9-076e-43c0-b9cc-66233f817f48 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:35:10.212779 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-315281c9-076e-43c0-b9cc-66233f817f48 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:35:10.213414 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-315281c9-076e-43c0-b9cc-66233f817f48 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:35:10.214016 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-315281c9-076e-43c0-b9cc-66233f817f48 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:35:10.275509 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-315281c9-076e-43c0-b9cc-66233f817f48 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:35:10.276248 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-315281c9-076e-43c0-b9cc-66233f817f48 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:35:10.276792 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-315281c9-076e-43c0-b9cc-66233f817f48 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:35:10.277385 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.objects.instance [None req-315281c9-076e-43c0-b9cc-66233f817f48 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lazy-loading 'fault' on Instance uuid fff5c233-ad7f-4022-8582-e8620f32ebd5 {{(pid=120864) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Aug 27 02:35:10.331769 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-315281c9-076e-43c0-b9cc-66233f817f48 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:35:10.340396 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-315281c9-076e-43c0-b9cc-66233f817f48 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:35:10.341948 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-315281c9-076e-43c0-b9cc-66233f817f48 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd750e6e172d34d4a9333e0c059c5442b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c938ae5fd22a43e8a01859da2a336b53', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:35:10.343279 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-315281c9-076e-43c0-b9cc-66233f817f48 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] 10.222.0.52 "GET /compute/v2.1/servers/fff5c233-ad7f-4022-8582-e8620f32ebd5" status: 200 len: 1372 microversion: 2.20 time: 0.151381 Aug 27 02:35:10.344621 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 519/1031] 10.222.0.52 () {60 vars in 1277 bytes} [Sat Aug 27 02:35:10 2022] GET /compute/v2.1/servers/fff5c233-ad7f-4022-8582-e8620f32ebd5 => generated 1372 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 27 02:35:11.369353 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-c23db6a2-c6e9-47b8-8819-8cac030128f3 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:35:11.370051 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-c23db6a2-c6e9-47b8-8819-8cac030128f3 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] [instance: fff5c233-ad7f-4022-8582-e8620f32ebd5] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:35:11.376433 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-c23db6a2-c6e9-47b8-8819-8cac030128f3 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:35:11.376930 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-c23db6a2-c6e9-47b8-8819-8cac030128f3 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:35:11.377389 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-c23db6a2-c6e9-47b8-8819-8cac030128f3 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:35:11.378094 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-c23db6a2-c6e9-47b8-8819-8cac030128f3 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:35:11.378548 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-c23db6a2-c6e9-47b8-8819-8cac030128f3 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:35:11.379173 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-c23db6a2-c6e9-47b8-8819-8cac030128f3 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:35:11.392611 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.wsgi [None req-c23db6a2-c6e9-47b8-8819-8cac030128f3 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] HTTP exception thrown: Instance fff5c233-ad7f-4022-8582-e8620f32ebd5 could not be found. Aug 27 02:35:11.394305 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-c23db6a2-c6e9-47b8-8819-8cac030128f3 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Request method failure captured: Aug 27 02:35:11.394305 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: request: GET /compute/v2.1/servers/fff5c233-ad7f-4022-8582-e8620f32ebd5 HTTP/1.1 Aug 27 02:35:11.394305 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: Accept: application/json Aug 27 02:35:11.394305 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: Accept-Encoding: identity Aug 27 02:35:11.394305 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: Connection: close Aug 27 02:35:11.394305 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: Content-Length: 0 Aug 27 02:35:11.394305 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: Content-Type: application/json Aug 27 02:35:11.394305 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: Host: 10.222.0.52 Aug 27 02:35:11.394305 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: Openstack-System-Scope: None Aug 27 02:35:11.394305 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: User-Agent: python-urllib3/1.26.11 Aug 27 02:35:11.394305 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-Auth-Token: gAAAAABjCYBiNuRGqbSfO7dm9FqSum3tt3k6TUULoLKrodG50GilNY4hLrqiBLkDm5s9EALB7jkOLV7WRs6Rnc2MYSkA5wTMgskawzqFmTTWFZKYym29LUjbwB2x6tewJf0ruhSgvMmooyOttIPsYVQZ7FYwUN1sl1AtWnQXmZA3HDwawNzSXec Aug 27 02:35:11.394305 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-Domain-Id: None Aug 27 02:35:11.394305 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-Domain-Name: None Aug 27 02:35:11.394305 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-Identity-Status: Confirmed Aug 27 02:35:11.394305 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-Is-Admin-Project: True Aug 27 02:35:11.394305 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-Openstack-Nova-Api-Version: 2.20 Aug 27 02:35:11.394305 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-Project-Domain-Id: default Aug 27 02:35:11.394305 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-Project-Domain-Name: Default Aug 27 02:35:11.394305 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-Project-Id: c938ae5fd22a43e8a01859da2a336b53 Aug 27 02:35:11.394305 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-Project-Name: tempest-AttachVolumeShelveTestJSON-363465878 Aug 27 02:35:11.394305 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-Role: reader,member Aug 27 02:35:11.394305 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-Roles: reader,member Aug 27 02:35:11.396180 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-Service-Catalog: [{"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.52:8080", "publicURL": "http://10.222.0.52:8080/v1/AUTH_c938ae5fd22a43e8a01859da2a336b53"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52:9696/networking"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/compute/v2/c938ae5fd22a43e8a01859da2a336b53"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/image"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/volume/v3/c938ae5fd22a43e8a01859da2a336b53"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/identity"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/volume/v3/c938ae5fd22a43e8a01859da2a336b53"}]}] Aug 27 02:35:11.396180 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-Tenant: tempest-AttachVolumeShelveTestJSON-363465878 Aug 27 02:35:11.396180 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-Tenant-Id: c938ae5fd22a43e8a01859da2a336b53 Aug 27 02:35:11.396180 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-Tenant-Name: tempest-AttachVolumeShelveTestJSON-363465878 Aug 27 02:35:11.396180 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-User: tempest-AttachVolumeShelveTestJSON-363465878-project Aug 27 02:35:11.396180 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-User-Domain-Id: default Aug 27 02:35:11.396180 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-User-Domain-Name: Default Aug 27 02:35:11.396180 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-User-Id: d750e6e172d34d4a9333e0c059c5442b Aug 27 02:35:11.396180 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-User-Name: tempest-AttachVolumeShelveTestJSON-363465878-project Aug 27 02:35:11.396180 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: method: > Aug 27 02:35:11.396180 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: exception: Instance fff5c233-ad7f-4022-8582-e8620f32ebd5 could not be found. Aug 27 02:35:11.396180 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: action_args: {'id': 'fff5c233-ad7f-4022-8582-e8620f32ebd5'} Aug 27 02:35:11.397037 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Aug 27 02:35:11.397037 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 27 02:35:11.397037 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Aug 27 02:35:11.397037 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Aug 27 02:35:11.397037 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Aug 27 02:35:11.397037 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Aug 27 02:35:11.397037 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Aug 27 02:35:11.397037 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Aug 27 02:35:11.397037 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Aug 27 02:35:11.397037 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Aug 27 02:35:11.397037 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance fff5c233-ad7f-4022-8582-e8620f32ebd5 could not be found. Aug 27 02:35:11.397037 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi Aug 27 02:35:11.397037 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 27 02:35:11.397037 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi Aug 27 02:35:11.397037 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 27 02:35:11.397037 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 27 02:35:11.397037 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 27 02:35:11.397037 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Aug 27 02:35:11.397037 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Aug 27 02:35:11.397037 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Aug 27 02:35:11.398730 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Aug 27 02:35:11.398730 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Aug 27 02:35:11.398730 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi instance = self._get_server( Aug 27 02:35:11.398730 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Aug 27 02:35:11.398730 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Aug 27 02:35:11.398730 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Aug 27 02:35:11.398730 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Aug 27 02:35:11.398730 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance fff5c233-ad7f-4022-8582-e8620f32ebd5 could not be found. Aug 27 02:35:11.398730 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi Aug 27 02:35:11.398730 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 27 02:35:11.398730 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi Aug 27 02:35:11.398730 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 27 02:35:11.398730 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 27 02:35:11.398730 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 27 02:35:11.398730 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Aug 27 02:35:11.398730 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Aug 27 02:35:11.398730 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance fff5c233-ad7f-4022-8582-e8620f32ebd5 could not be found. Aug 27 02:35:11.398730 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi Aug 27 02:35:11.403451 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-c23db6a2-c6e9-47b8-8819-8cac030128f3 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Returning 404 to user: Instance fff5c233-ad7f-4022-8582-e8620f32ebd5 could not be found. {{(pid=120863) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Aug 27 02:35:11.403451 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-c23db6a2-c6e9-47b8-8819-8cac030128f3 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] 10.222.0.52 "GET /compute/v2.1/servers/fff5c233-ad7f-4022-8582-e8620f32ebd5" status: 404 len: 111 microversion: 2.20 time: 0.031346 Aug 27 02:35:11.403451 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 513/1032] 10.222.0.52 () {60 vars in 1277 bytes} [Sat Aug 27 02:35:11 2022] GET /compute/v2.1/servers/fff5c233-ad7f-4022-8582-e8620f32ebd5 => generated 111 bytes in 38 msecs (HTTP/1.1 404) 9 headers in 381 bytes (1 switches on core 0) Aug 27 02:35:11.409535 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-1279c727-8660-48c4-b936-61dec81d4988 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Calling method '.version_select at 0x7f6b1e7db5e0>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:35:11.431967 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-1279c727-8660-48c4-b936-61dec81d4988 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] 10.222.0.52 "DELETE /compute/v2.1/os-keypairs/tempest-keypair-2023400689" status: 204 len: 0 microversion: 2.20 time: 0.026138 Aug 27 02:35:11.432557 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 520/1033] 10.222.0.52 () {60 vars in 1262 bytes} [Sat Aug 27 02:35:11 2022] DELETE /compute/v2.1/os-keypairs/tempest-keypair-2023400689 => generated 0 bytes in 28 msecs (HTTP/1.1 204) 9 headers in 365 bytes (1 switches on core 0) Aug 27 02:35:13.276081 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-67eb3a73-e94f-4cb8-a379-798594e9beca tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:35:13.276612 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-67eb3a73-e94f-4cb8-a379-798594e9beca tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] [instance: fff5c233-ad7f-4022-8582-e8620f32ebd5] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:35:13.283664 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-67eb3a73-e94f-4cb8-a379-798594e9beca tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:35:13.284143 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-67eb3a73-e94f-4cb8-a379-798594e9beca tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:35:13.284572 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-67eb3a73-e94f-4cb8-a379-798594e9beca tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:35:13.285300 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-67eb3a73-e94f-4cb8-a379-798594e9beca tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:35:13.285731 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-67eb3a73-e94f-4cb8-a379-798594e9beca tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:35:13.286226 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-67eb3a73-e94f-4cb8-a379-798594e9beca tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:35:13.300490 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.wsgi [None req-67eb3a73-e94f-4cb8-a379-798594e9beca tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] HTTP exception thrown: Instance fff5c233-ad7f-4022-8582-e8620f32ebd5 could not be found. Aug 27 02:35:13.302400 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-67eb3a73-e94f-4cb8-a379-798594e9beca tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Request method failure captured: Aug 27 02:35:13.302400 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: request: DELETE /compute/v2.1/servers/fff5c233-ad7f-4022-8582-e8620f32ebd5 HTTP/1.1 Aug 27 02:35:13.302400 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: Accept: application/json Aug 27 02:35:13.302400 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: Accept-Encoding: identity Aug 27 02:35:13.302400 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: Connection: close Aug 27 02:35:13.302400 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: Content-Length: 0 Aug 27 02:35:13.302400 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: Content-Type: application/json Aug 27 02:35:13.302400 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: Host: 10.222.0.52 Aug 27 02:35:13.302400 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: Openstack-System-Scope: None Aug 27 02:35:13.302400 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: User-Agent: python-urllib3/1.26.11 Aug 27 02:35:13.302400 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-Auth-Token: gAAAAABjCYBiNuRGqbSfO7dm9FqSum3tt3k6TUULoLKrodG50GilNY4hLrqiBLkDm5s9EALB7jkOLV7WRs6Rnc2MYSkA5wTMgskawzqFmTTWFZKYym29LUjbwB2x6tewJf0ruhSgvMmooyOttIPsYVQZ7FYwUN1sl1AtWnQXmZA3HDwawNzSXec Aug 27 02:35:13.302400 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-Domain-Id: None Aug 27 02:35:13.302400 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-Domain-Name: None Aug 27 02:35:13.302400 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-Identity-Status: Confirmed Aug 27 02:35:13.302400 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-Is-Admin-Project: True Aug 27 02:35:13.302400 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-Openstack-Nova-Api-Version: 2.20 Aug 27 02:35:13.302400 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-Project-Domain-Id: default Aug 27 02:35:13.302400 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-Project-Domain-Name: Default Aug 27 02:35:13.302400 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-Project-Id: c938ae5fd22a43e8a01859da2a336b53 Aug 27 02:35:13.302400 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-Project-Name: tempest-AttachVolumeShelveTestJSON-363465878 Aug 27 02:35:13.302400 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-Role: reader,member Aug 27 02:35:13.302400 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-Roles: reader,member Aug 27 02:35:13.304095 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-Service-Catalog: [{"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.52:8080", "publicURL": "http://10.222.0.52:8080/v1/AUTH_c938ae5fd22a43e8a01859da2a336b53"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52:9696/networking"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/compute/v2/c938ae5fd22a43e8a01859da2a336b53"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/image"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/volume/v3/c938ae5fd22a43e8a01859da2a336b53"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/identity"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/volume/v3/c938ae5fd22a43e8a01859da2a336b53"}]}] Aug 27 02:35:13.304095 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-Tenant: tempest-AttachVolumeShelveTestJSON-363465878 Aug 27 02:35:13.304095 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-Tenant-Id: c938ae5fd22a43e8a01859da2a336b53 Aug 27 02:35:13.304095 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-Tenant-Name: tempest-AttachVolumeShelveTestJSON-363465878 Aug 27 02:35:13.304095 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-User: tempest-AttachVolumeShelveTestJSON-363465878-project Aug 27 02:35:13.304095 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-User-Domain-Id: default Aug 27 02:35:13.304095 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-User-Domain-Name: Default Aug 27 02:35:13.304095 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-User-Id: d750e6e172d34d4a9333e0c059c5442b Aug 27 02:35:13.304095 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-User-Name: tempest-AttachVolumeShelveTestJSON-363465878-project Aug 27 02:35:13.304095 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: method: > Aug 27 02:35:13.304095 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: exception: Instance fff5c233-ad7f-4022-8582-e8620f32ebd5 could not be found. Aug 27 02:35:13.304095 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: action_args: {'id': 'fff5c233-ad7f-4022-8582-e8620f32ebd5'} Aug 27 02:35:13.304894 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Aug 27 02:35:13.304894 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 27 02:35:13.304894 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Aug 27 02:35:13.304894 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Aug 27 02:35:13.304894 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Aug 27 02:35:13.304894 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Aug 27 02:35:13.304894 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Aug 27 02:35:13.304894 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Aug 27 02:35:13.304894 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Aug 27 02:35:13.304894 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Aug 27 02:35:13.304894 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance fff5c233-ad7f-4022-8582-e8620f32ebd5 could not be found. Aug 27 02:35:13.304894 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi Aug 27 02:35:13.304894 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 27 02:35:13.304894 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi Aug 27 02:35:13.304894 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 27 02:35:13.304894 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 27 02:35:13.304894 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 27 02:35:13.304894 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Aug 27 02:35:13.304894 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Aug 27 02:35:13.304894 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Aug 27 02:35:13.306248 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Aug 27 02:35:13.306248 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Aug 27 02:35:13.306248 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Aug 27 02:35:13.306248 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Aug 27 02:35:13.306248 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Aug 27 02:35:13.306248 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Aug 27 02:35:13.306248 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Aug 27 02:35:13.306248 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Aug 27 02:35:13.306248 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Aug 27 02:35:13.306248 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance fff5c233-ad7f-4022-8582-e8620f32ebd5 could not be found. Aug 27 02:35:13.306248 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi Aug 27 02:35:13.306248 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 27 02:35:13.306248 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi Aug 27 02:35:13.306248 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 27 02:35:13.306248 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 27 02:35:13.306248 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 27 02:35:13.306248 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Aug 27 02:35:13.306248 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Aug 27 02:35:13.306248 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance fff5c233-ad7f-4022-8582-e8620f32ebd5 could not be found. Aug 27 02:35:13.306248 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi Aug 27 02:35:13.307536 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-67eb3a73-e94f-4cb8-a379-798594e9beca tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Returning 404 to user: Instance fff5c233-ad7f-4022-8582-e8620f32ebd5 could not be found. {{(pid=120863) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Aug 27 02:35:13.307536 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-67eb3a73-e94f-4cb8-a379-798594e9beca tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] 10.222.0.52 "DELETE /compute/v2.1/servers/fff5c233-ad7f-4022-8582-e8620f32ebd5" status: 404 len: 111 microversion: 2.20 time: 0.031810 Aug 27 02:35:13.307536 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 514/1034] 10.222.0.52 () {60 vars in 1280 bytes} [Sat Aug 27 02:35:13 2022] DELETE /compute/v2.1/servers/fff5c233-ad7f-4022-8582-e8620f32ebd5 => generated 111 bytes in 33 msecs (HTTP/1.1 404) 9 headers in 381 bytes (1 switches on core 0) Aug 27 02:35:13.313009 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-07e62e9e-0af7-4475-b48a-c625f33f7d1a tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Calling method '>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:35:13.313651 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-07e62e9e-0af7-4475-b48a-c625f33f7d1a tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] [instance: fff5c233-ad7f-4022-8582-e8620f32ebd5] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:35:13.324376 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-07e62e9e-0af7-4475-b48a-c625f33f7d1a tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:35:13.324865 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-07e62e9e-0af7-4475-b48a-c625f33f7d1a tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:35:13.325344 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-07e62e9e-0af7-4475-b48a-c625f33f7d1a tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:35:13.326046 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-07e62e9e-0af7-4475-b48a-c625f33f7d1a tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:35:13.326501 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-07e62e9e-0af7-4475-b48a-c625f33f7d1a tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:35:13.326925 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-07e62e9e-0af7-4475-b48a-c625f33f7d1a tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:35:13.341099 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.wsgi [None req-07e62e9e-0af7-4475-b48a-c625f33f7d1a tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] HTTP exception thrown: Instance fff5c233-ad7f-4022-8582-e8620f32ebd5 could not be found. Aug 27 02:35:13.342590 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-07e62e9e-0af7-4475-b48a-c625f33f7d1a tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Request method failure captured: Aug 27 02:35:13.342590 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: request: GET /compute/v2.1/servers/fff5c233-ad7f-4022-8582-e8620f32ebd5 HTTP/1.1 Aug 27 02:35:13.342590 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: Accept: application/json Aug 27 02:35:13.342590 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: Accept-Encoding: identity Aug 27 02:35:13.342590 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: Connection: close Aug 27 02:35:13.342590 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: Content-Length: 0 Aug 27 02:35:13.342590 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: Content-Type: application/json Aug 27 02:35:13.342590 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: Host: 10.222.0.52 Aug 27 02:35:13.342590 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: Openstack-System-Scope: None Aug 27 02:35:13.342590 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: User-Agent: python-urllib3/1.26.11 Aug 27 02:35:13.342590 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-Auth-Token: gAAAAABjCYBiNuRGqbSfO7dm9FqSum3tt3k6TUULoLKrodG50GilNY4hLrqiBLkDm5s9EALB7jkOLV7WRs6Rnc2MYSkA5wTMgskawzqFmTTWFZKYym29LUjbwB2x6tewJf0ruhSgvMmooyOttIPsYVQZ7FYwUN1sl1AtWnQXmZA3HDwawNzSXec Aug 27 02:35:13.342590 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-Domain-Id: None Aug 27 02:35:13.342590 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-Domain-Name: None Aug 27 02:35:13.342590 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-Identity-Status: Confirmed Aug 27 02:35:13.342590 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-Is-Admin-Project: True Aug 27 02:35:13.342590 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-Openstack-Nova-Api-Version: 2.20 Aug 27 02:35:13.342590 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-Project-Domain-Id: default Aug 27 02:35:13.342590 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-Project-Domain-Name: Default Aug 27 02:35:13.342590 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-Project-Id: c938ae5fd22a43e8a01859da2a336b53 Aug 27 02:35:13.342590 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-Project-Name: tempest-AttachVolumeShelveTestJSON-363465878 Aug 27 02:35:13.342590 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-Role: reader,member Aug 27 02:35:13.342590 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-Roles: reader,member Aug 27 02:35:13.344296 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-Service-Catalog: [{"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.52:8080", "publicURL": "http://10.222.0.52:8080/v1/AUTH_c938ae5fd22a43e8a01859da2a336b53"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52:9696/networking"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/compute/v2/c938ae5fd22a43e8a01859da2a336b53"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/image"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/volume/v3/c938ae5fd22a43e8a01859da2a336b53"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/identity"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/volume/v3/c938ae5fd22a43e8a01859da2a336b53"}]}] Aug 27 02:35:13.344296 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-Tenant: tempest-AttachVolumeShelveTestJSON-363465878 Aug 27 02:35:13.344296 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-Tenant-Id: c938ae5fd22a43e8a01859da2a336b53 Aug 27 02:35:13.344296 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-Tenant-Name: tempest-AttachVolumeShelveTestJSON-363465878 Aug 27 02:35:13.344296 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-User: tempest-AttachVolumeShelveTestJSON-363465878-project Aug 27 02:35:13.344296 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-User-Domain-Id: default Aug 27 02:35:13.344296 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-User-Domain-Name: Default Aug 27 02:35:13.344296 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-User-Id: d750e6e172d34d4a9333e0c059c5442b Aug 27 02:35:13.344296 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-User-Name: tempest-AttachVolumeShelveTestJSON-363465878-project Aug 27 02:35:13.344296 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: method: > Aug 27 02:35:13.344296 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: exception: Instance fff5c233-ad7f-4022-8582-e8620f32ebd5 could not be found. Aug 27 02:35:13.344296 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: action_args: {'id': 'fff5c233-ad7f-4022-8582-e8620f32ebd5'} Aug 27 02:35:13.345235 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Aug 27 02:35:13.345235 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 27 02:35:13.345235 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Aug 27 02:35:13.345235 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Aug 27 02:35:13.345235 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Aug 27 02:35:13.345235 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Aug 27 02:35:13.345235 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Aug 27 02:35:13.345235 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Aug 27 02:35:13.345235 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Aug 27 02:35:13.345235 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Aug 27 02:35:13.345235 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance fff5c233-ad7f-4022-8582-e8620f32ebd5 could not be found. Aug 27 02:35:13.345235 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi Aug 27 02:35:13.345235 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 27 02:35:13.345235 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi Aug 27 02:35:13.345235 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 27 02:35:13.345235 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 27 02:35:13.345235 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 27 02:35:13.345235 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Aug 27 02:35:13.345235 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Aug 27 02:35:13.345235 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Aug 27 02:35:13.346762 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Aug 27 02:35:13.346762 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Aug 27 02:35:13.346762 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi instance = self._get_server( Aug 27 02:35:13.346762 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Aug 27 02:35:13.346762 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Aug 27 02:35:13.346762 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Aug 27 02:35:13.346762 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Aug 27 02:35:13.346762 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance fff5c233-ad7f-4022-8582-e8620f32ebd5 could not be found. Aug 27 02:35:13.346762 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi Aug 27 02:35:13.346762 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 27 02:35:13.346762 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi Aug 27 02:35:13.346762 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 27 02:35:13.346762 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 27 02:35:13.346762 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 27 02:35:13.346762 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Aug 27 02:35:13.346762 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Aug 27 02:35:13.346762 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance fff5c233-ad7f-4022-8582-e8620f32ebd5 could not be found. Aug 27 02:35:13.346762 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi Aug 27 02:35:13.348393 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-07e62e9e-0af7-4475-b48a-c625f33f7d1a tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Returning 404 to user: Instance fff5c233-ad7f-4022-8582-e8620f32ebd5 could not be found. {{(pid=120864) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Aug 27 02:35:13.348393 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-07e62e9e-0af7-4475-b48a-c625f33f7d1a tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] 10.222.0.52 "GET /compute/v2.1/servers/fff5c233-ad7f-4022-8582-e8620f32ebd5" status: 404 len: 111 microversion: 2.20 time: 0.035458 Aug 27 02:35:13.348393 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 521/1035] 10.222.0.52 () {60 vars in 1277 bytes} [Sat Aug 27 02:35:13 2022] GET /compute/v2.1/servers/fff5c233-ad7f-4022-8582-e8620f32ebd5 => generated 111 bytes in 36 msecs (HTTP/1.1 404) 9 headers in 381 bytes (1 switches on core 0) Aug 27 02:35:17.681930 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-fe5bb0d5-bc7a-472a-a333-ee4d55a3f075 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:35:17.682729 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-fe5bb0d5-bc7a-472a-a333-ee4d55a3f075 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] [instance: 56dbfa6a-5b6f-4b82-977b-af735e6a6ac9] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:35:17.690699 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-fe5bb0d5-bc7a-472a-a333-ee4d55a3f075 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:35:17.691178 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-fe5bb0d5-bc7a-472a-a333-ee4d55a3f075 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:35:17.691656 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-fe5bb0d5-bc7a-472a-a333-ee4d55a3f075 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:35:17.692687 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-fe5bb0d5-bc7a-472a-a333-ee4d55a3f075 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:35:17.693171 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-fe5bb0d5-bc7a-472a-a333-ee4d55a3f075 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:35:17.693601 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-fe5bb0d5-bc7a-472a-a333-ee4d55a3f075 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:35:17.709149 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.wsgi [None req-fe5bb0d5-bc7a-472a-a333-ee4d55a3f075 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] HTTP exception thrown: Instance 56dbfa6a-5b6f-4b82-977b-af735e6a6ac9 could not be found. Aug 27 02:35:17.711157 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-fe5bb0d5-bc7a-472a-a333-ee4d55a3f075 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Request method failure captured: Aug 27 02:35:17.711157 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: request: DELETE /compute/v2.1/servers/56dbfa6a-5b6f-4b82-977b-af735e6a6ac9 HTTP/1.1 Aug 27 02:35:17.711157 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: Accept: application/json Aug 27 02:35:17.711157 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: Accept-Encoding: identity Aug 27 02:35:17.711157 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: Connection: close Aug 27 02:35:17.711157 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: Content-Length: 0 Aug 27 02:35:17.711157 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: Content-Type: application/json Aug 27 02:35:17.711157 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: Host: 10.222.0.52 Aug 27 02:35:17.711157 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: Openstack-System-Scope: None Aug 27 02:35:17.711157 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: User-Agent: python-urllib3/1.26.11 Aug 27 02:35:17.711157 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-Auth-Token: gAAAAABjCYBiNuRGqbSfO7dm9FqSum3tt3k6TUULoLKrodG50GilNY4hLrqiBLkDm5s9EALB7jkOLV7WRs6Rnc2MYSkA5wTMgskawzqFmTTWFZKYym29LUjbwB2x6tewJf0ruhSgvMmooyOttIPsYVQZ7FYwUN1sl1AtWnQXmZA3HDwawNzSXec Aug 27 02:35:17.711157 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-Domain-Id: None Aug 27 02:35:17.711157 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-Domain-Name: None Aug 27 02:35:17.711157 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-Identity-Status: Confirmed Aug 27 02:35:17.711157 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-Is-Admin-Project: True Aug 27 02:35:17.711157 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-Openstack-Nova-Api-Version: 2.20 Aug 27 02:35:17.711157 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-Project-Domain-Id: default Aug 27 02:35:17.711157 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-Project-Domain-Name: Default Aug 27 02:35:17.711157 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-Project-Id: c938ae5fd22a43e8a01859da2a336b53 Aug 27 02:35:17.711157 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-Project-Name: tempest-AttachVolumeShelveTestJSON-363465878 Aug 27 02:35:17.711157 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-Role: reader,member Aug 27 02:35:17.711157 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-Roles: reader,member Aug 27 02:35:17.712986 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-Service-Catalog: [{"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.52:8080", "publicURL": "http://10.222.0.52:8080/v1/AUTH_c938ae5fd22a43e8a01859da2a336b53"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52:9696/networking"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/compute/v2/c938ae5fd22a43e8a01859da2a336b53"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/image"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/volume/v3/c938ae5fd22a43e8a01859da2a336b53"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/identity"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/volume/v3/c938ae5fd22a43e8a01859da2a336b53"}]}] Aug 27 02:35:17.712986 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-Tenant: tempest-AttachVolumeShelveTestJSON-363465878 Aug 27 02:35:17.712986 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-Tenant-Id: c938ae5fd22a43e8a01859da2a336b53 Aug 27 02:35:17.712986 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-Tenant-Name: tempest-AttachVolumeShelveTestJSON-363465878 Aug 27 02:35:17.712986 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-User: tempest-AttachVolumeShelveTestJSON-363465878-project Aug 27 02:35:17.712986 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-User-Domain-Id: default Aug 27 02:35:17.712986 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-User-Domain-Name: Default Aug 27 02:35:17.712986 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-User-Id: d750e6e172d34d4a9333e0c059c5442b Aug 27 02:35:17.712986 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-User-Name: tempest-AttachVolumeShelveTestJSON-363465878-project Aug 27 02:35:17.712986 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: method: > Aug 27 02:35:17.712986 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: exception: Instance 56dbfa6a-5b6f-4b82-977b-af735e6a6ac9 could not be found. Aug 27 02:35:17.712986 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: action_args: {'id': '56dbfa6a-5b6f-4b82-977b-af735e6a6ac9'} Aug 27 02:35:17.714127 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Aug 27 02:35:17.714127 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 27 02:35:17.714127 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Aug 27 02:35:17.714127 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Aug 27 02:35:17.714127 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Aug 27 02:35:17.714127 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Aug 27 02:35:17.714127 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Aug 27 02:35:17.714127 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Aug 27 02:35:17.714127 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Aug 27 02:35:17.714127 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Aug 27 02:35:17.714127 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 56dbfa6a-5b6f-4b82-977b-af735e6a6ac9 could not be found. Aug 27 02:35:17.714127 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi Aug 27 02:35:17.714127 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 27 02:35:17.714127 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi Aug 27 02:35:17.714127 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 27 02:35:17.714127 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 27 02:35:17.714127 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 27 02:35:17.714127 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Aug 27 02:35:17.714127 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Aug 27 02:35:17.714127 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Aug 27 02:35:17.715543 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Aug 27 02:35:17.715543 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Aug 27 02:35:17.715543 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Aug 27 02:35:17.715543 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Aug 27 02:35:17.715543 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Aug 27 02:35:17.715543 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Aug 27 02:35:17.715543 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Aug 27 02:35:17.715543 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Aug 27 02:35:17.715543 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Aug 27 02:35:17.715543 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 56dbfa6a-5b6f-4b82-977b-af735e6a6ac9 could not be found. Aug 27 02:35:17.715543 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi Aug 27 02:35:17.715543 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 27 02:35:17.715543 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi Aug 27 02:35:17.715543 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 27 02:35:17.715543 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 27 02:35:17.715543 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 27 02:35:17.715543 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Aug 27 02:35:17.715543 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Aug 27 02:35:17.715543 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 56dbfa6a-5b6f-4b82-977b-af735e6a6ac9 could not be found. Aug 27 02:35:17.715543 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi Aug 27 02:35:17.716976 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-fe5bb0d5-bc7a-472a-a333-ee4d55a3f075 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Returning 404 to user: Instance 56dbfa6a-5b6f-4b82-977b-af735e6a6ac9 could not be found. {{(pid=120863) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Aug 27 02:35:17.716976 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-fe5bb0d5-bc7a-472a-a333-ee4d55a3f075 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] 10.222.0.52 "DELETE /compute/v2.1/servers/56dbfa6a-5b6f-4b82-977b-af735e6a6ac9" status: 404 len: 111 microversion: 2.20 time: 0.036167 Aug 27 02:35:17.716976 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 515/1036] 10.222.0.52 () {60 vars in 1280 bytes} [Sat Aug 27 02:35:17 2022] DELETE /compute/v2.1/servers/56dbfa6a-5b6f-4b82-977b-af735e6a6ac9 => generated 111 bytes in 37 msecs (HTTP/1.1 404) 9 headers in 381 bytes (1 switches on core 0) Aug 27 02:35:17.722764 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-23844c6d-5b2d-461c-8936-713bed3a5b49 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Calling method '>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:35:17.723436 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-23844c6d-5b2d-461c-8936-713bed3a5b49 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] [instance: 56dbfa6a-5b6f-4b82-977b-af735e6a6ac9] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:35:17.731014 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-23844c6d-5b2d-461c-8936-713bed3a5b49 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:35:17.731506 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-23844c6d-5b2d-461c-8936-713bed3a5b49 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:35:17.731940 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-23844c6d-5b2d-461c-8936-713bed3a5b49 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:35:17.732637 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-23844c6d-5b2d-461c-8936-713bed3a5b49 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:35:17.733090 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-23844c6d-5b2d-461c-8936-713bed3a5b49 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:35:17.733546 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-23844c6d-5b2d-461c-8936-713bed3a5b49 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:35:17.749259 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.wsgi [None req-23844c6d-5b2d-461c-8936-713bed3a5b49 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] HTTP exception thrown: Instance 56dbfa6a-5b6f-4b82-977b-af735e6a6ac9 could not be found. Aug 27 02:35:17.750896 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-23844c6d-5b2d-461c-8936-713bed3a5b49 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Request method failure captured: Aug 27 02:35:17.750896 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: request: GET /compute/v2.1/servers/56dbfa6a-5b6f-4b82-977b-af735e6a6ac9 HTTP/1.1 Aug 27 02:35:17.750896 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: Accept: application/json Aug 27 02:35:17.750896 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: Accept-Encoding: identity Aug 27 02:35:17.750896 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: Connection: close Aug 27 02:35:17.750896 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: Content-Length: 0 Aug 27 02:35:17.750896 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: Content-Type: application/json Aug 27 02:35:17.750896 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: Host: 10.222.0.52 Aug 27 02:35:17.750896 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: Openstack-System-Scope: None Aug 27 02:35:17.750896 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: User-Agent: python-urllib3/1.26.11 Aug 27 02:35:17.750896 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-Auth-Token: gAAAAABjCYBiNuRGqbSfO7dm9FqSum3tt3k6TUULoLKrodG50GilNY4hLrqiBLkDm5s9EALB7jkOLV7WRs6Rnc2MYSkA5wTMgskawzqFmTTWFZKYym29LUjbwB2x6tewJf0ruhSgvMmooyOttIPsYVQZ7FYwUN1sl1AtWnQXmZA3HDwawNzSXec Aug 27 02:35:17.750896 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-Domain-Id: None Aug 27 02:35:17.750896 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-Domain-Name: None Aug 27 02:35:17.750896 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-Identity-Status: Confirmed Aug 27 02:35:17.750896 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-Is-Admin-Project: True Aug 27 02:35:17.750896 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-Openstack-Nova-Api-Version: 2.20 Aug 27 02:35:17.750896 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-Project-Domain-Id: default Aug 27 02:35:17.750896 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-Project-Domain-Name: Default Aug 27 02:35:17.750896 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-Project-Id: c938ae5fd22a43e8a01859da2a336b53 Aug 27 02:35:17.750896 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-Project-Name: tempest-AttachVolumeShelveTestJSON-363465878 Aug 27 02:35:17.750896 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-Role: reader,member Aug 27 02:35:17.750896 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-Roles: reader,member Aug 27 02:35:17.752530 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-Service-Catalog: [{"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.52:8080", "publicURL": "http://10.222.0.52:8080/v1/AUTH_c938ae5fd22a43e8a01859da2a336b53"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52:9696/networking"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/compute/v2/c938ae5fd22a43e8a01859da2a336b53"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/image"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/volume/v3/c938ae5fd22a43e8a01859da2a336b53"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/identity"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/volume/v3/c938ae5fd22a43e8a01859da2a336b53"}]}] Aug 27 02:35:17.752530 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-Tenant: tempest-AttachVolumeShelveTestJSON-363465878 Aug 27 02:35:17.752530 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-Tenant-Id: c938ae5fd22a43e8a01859da2a336b53 Aug 27 02:35:17.752530 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-Tenant-Name: tempest-AttachVolumeShelveTestJSON-363465878 Aug 27 02:35:17.752530 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-User: tempest-AttachVolumeShelveTestJSON-363465878-project Aug 27 02:35:17.752530 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-User-Domain-Id: default Aug 27 02:35:17.752530 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-User-Domain-Name: Default Aug 27 02:35:17.752530 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-User-Id: d750e6e172d34d4a9333e0c059c5442b Aug 27 02:35:17.752530 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-User-Name: tempest-AttachVolumeShelveTestJSON-363465878-project Aug 27 02:35:17.752530 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: method: > Aug 27 02:35:17.752530 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: exception: Instance 56dbfa6a-5b6f-4b82-977b-af735e6a6ac9 could not be found. Aug 27 02:35:17.752530 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: action_args: {'id': '56dbfa6a-5b6f-4b82-977b-af735e6a6ac9'} Aug 27 02:35:17.753381 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Aug 27 02:35:17.753381 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 27 02:35:17.753381 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Aug 27 02:35:17.753381 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Aug 27 02:35:17.753381 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Aug 27 02:35:17.753381 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Aug 27 02:35:17.753381 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Aug 27 02:35:17.753381 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Aug 27 02:35:17.753381 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Aug 27 02:35:17.753381 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Aug 27 02:35:17.753381 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 56dbfa6a-5b6f-4b82-977b-af735e6a6ac9 could not be found. Aug 27 02:35:17.753381 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi Aug 27 02:35:17.753381 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 27 02:35:17.753381 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi Aug 27 02:35:17.753381 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 27 02:35:17.753381 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 27 02:35:17.753381 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 27 02:35:17.753381 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Aug 27 02:35:17.753381 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Aug 27 02:35:17.753381 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Aug 27 02:35:17.755200 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Aug 27 02:35:17.755200 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Aug 27 02:35:17.755200 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi instance = self._get_server( Aug 27 02:35:17.755200 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Aug 27 02:35:17.755200 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Aug 27 02:35:17.755200 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Aug 27 02:35:17.755200 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Aug 27 02:35:17.755200 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 56dbfa6a-5b6f-4b82-977b-af735e6a6ac9 could not be found. Aug 27 02:35:17.755200 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi Aug 27 02:35:17.755200 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 27 02:35:17.755200 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi Aug 27 02:35:17.755200 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 27 02:35:17.755200 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 27 02:35:17.755200 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 27 02:35:17.755200 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Aug 27 02:35:17.755200 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Aug 27 02:35:17.755200 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 56dbfa6a-5b6f-4b82-977b-af735e6a6ac9 could not be found. Aug 27 02:35:17.755200 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi Aug 27 02:35:17.756520 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-23844c6d-5b2d-461c-8936-713bed3a5b49 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] Returning 404 to user: Instance 56dbfa6a-5b6f-4b82-977b-af735e6a6ac9 could not be found. {{(pid=120864) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Aug 27 02:35:17.756520 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-23844c6d-5b2d-461c-8936-713bed3a5b49 tempest-AttachVolumeShelveTestJSON-363465878 tempest-AttachVolumeShelveTestJSON-363465878-project] 10.222.0.52 "GET /compute/v2.1/servers/56dbfa6a-5b6f-4b82-977b-af735e6a6ac9" status: 404 len: 111 microversion: 2.20 time: 0.034957 Aug 27 02:35:17.756520 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 522/1037] 10.222.0.52 () {60 vars in 1277 bytes} [Sat Aug 27 02:35:17 2022] GET /compute/v2.1/servers/56dbfa6a-5b6f-4b82-977b-af735e6a6ac9 => generated 111 bytes in 36 msecs (HTTP/1.1 404) 9 headers in 381 bytes (1 switches on core 0) Aug 27 02:35:29.395747 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG dbcounter [-] [120863] Writing DB stats nova_api:SELECT=17,nova_api:UPDATE=1 {{(pid=120863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 27 02:35:29.397058 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG dbcounter [-] [120863] Writing DB stats nova_cell1:SELECT=28,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=120863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 27 02:35:29.619281 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-8e8a2c8e-9444-4760-ace0-328af861c603 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Action: 'create', calling method: .version_select at 0x7f6b1b097ee0>, body: {"keypair": {"name": "tempest-keypair-1652285511", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBIYlumSzrzBuuOMEbu+7bbeNTwXofn6xuG4V94CcblzREfmcUxWVaRFHR7tpWAN6BowQLqDUerJeK4WWgJjmAKvGMkUSAWi9OtYGKb9NHiuWDsBisD7AVNsc0yuQWv6c/g=="}} {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 27 02:35:29.638795 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.quota [None req-8e8a2c8e-9444-4760-ace0-328af861c603 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Getting quotas for project a8055f4326f640fc8927f5cfdef39dd3. Resources: {'key_pairs'} {{(pid=120863) _get_quotas /opt/stack/nova/nova/quota.py:393}} Aug 27 02:35:29.643818 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.quota [None req-8e8a2c8e-9444-4760-ace0-328af861c603 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Getting quotas for user ee6084fffc544dfe8f1a9a30ae06be69 and project a8055f4326f640fc8927f5cfdef39dd3. Resources: {'key_pairs'} {{(pid=120863) _get_quotas /opt/stack/nova/nova/quota.py:383}} Aug 27 02:35:29.680334 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-8e8a2c8e-9444-4760-ace0-328af861c603 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] 10.222.0.52 "POST /compute/v2.1/os-keypairs" status: 200 len: 386 microversion: 2.1 time: 0.290147 Aug 27 02:35:29.683303 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 516/1038] 10.222.0.52 () {60 vars in 1159 bytes} [Sat Aug 27 02:35:29 2022] POST /compute/v2.1/os-keypairs => generated 386 bytes in 294 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 27 02:35:31.905429 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-460367b9-7374-4775-b043-05e47075f4c6 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-AttachVolumeTestJSON-server-393011441", "imageRef": "9fc30b36-b827-46f9-892d-30e808bc2b8a", "flavorRef": "452", "adminPass": "***", "networks": [{"uuid": "59638dff-2144-4bc2-a657-3cc12e54e5c1"}], "security_groups": [{"name": "tempest-securitygroup--1981273721"}], "key_name": "tempest-keypair-1652285511", "user_data": "}} {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 27 02:35:31.914965 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG dbcounter [-] [120864] Writing DB stats nova_api:SELECT=17,nova_api:DELETE=1 {{(pid=120864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 27 02:35:31.915782 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG dbcounter [-] [120864] Writing DB stats nova_cell1:SELECT=30 {{(pid=120864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 27 02:35:32.071213 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.network.neutron [None req-460367b9-7374-4775-b043-05e47075f4c6 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] validate_networks() for [('59638dff-2144-4bc2-a657-3cc12e54e5c1', None, None, None, None, None)] {{(pid=120864) validate_networks /opt/stack/nova/nova/network/neutron.py:2626}} Aug 27 02:35:33.567011 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.virt.hardware [None req-460367b9-7374-4775-b043-05e47075f4c6 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Flavor limits 0:0:0 {{(pid=120864) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Aug 27 02:35:33.569765 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.virt.hardware [None req-460367b9-7374-4775-b043-05e47075f4c6 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Image limits 0:0:0 {{(pid=120864) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Aug 27 02:35:33.570581 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.virt.hardware [None req-460367b9-7374-4775-b043-05e47075f4c6 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Flavor pref 0:0:0 {{(pid=120864) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Aug 27 02:35:33.571098 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.virt.hardware [None req-460367b9-7374-4775-b043-05e47075f4c6 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Image pref 0:0:0 {{(pid=120864) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Aug 27 02:35:33.571623 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.virt.hardware [None req-460367b9-7374-4775-b043-05e47075f4c6 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=120864) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Aug 27 02:35:33.587814 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-460367b9-7374-4775-b043-05e47075f4c6 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:35:33.589765 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-460367b9-7374-4775-b043-05e47075f4c6 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:35:33.590217 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-460367b9-7374-4775-b043-05e47075f4c6 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:35:33.591007 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-460367b9-7374-4775-b043-05e47075f4c6 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:35:33.591466 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-460367b9-7374-4775-b043-05e47075f4c6 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:35:33.591908 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-460367b9-7374-4775-b043-05e47075f4c6 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:35:33.614835 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.quota [None req-460367b9-7374-4775-b043-05e47075f4c6 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Getting quotas for project a8055f4326f640fc8927f5cfdef39dd3. Resources: {'cores', 'ram', 'instances'} {{(pid=120864) _get_quotas /opt/stack/nova/nova/quota.py:393}} Aug 27 02:35:33.620089 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.quota [None req-460367b9-7374-4775-b043-05e47075f4c6 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Getting quotas for user ee6084fffc544dfe8f1a9a30ae06be69 and project a8055f4326f640fc8927f5cfdef39dd3. Resources: {'cores', 'ram', 'instances'} {{(pid=120864) _get_quotas /opt/stack/nova/nova/quota.py:383}} Aug 27 02:35:33.634415 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-460367b9-7374-4775-b043-05e47075f4c6 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Going to run 1 instances... {{(pid=120864) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Aug 27 02:35:33.637158 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-460367b9-7374-4775-b043-05e47075f4c6 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] [instance: f5bad70c-350c-4230-a862-626dd6805e4c] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='9fc30b36-b827-46f9-892d-30e808bc2b8a',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=120864) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Aug 27 02:35:33.693156 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-460367b9-7374-4775-b043-05e47075f4c6 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] 10.222.0.52 "POST /compute/v2.1/servers" status: 202 len: 402 microversion: 2.1 time: 1.792600 Aug 27 02:35:33.700521 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 523/1039] 10.222.0.52 () {60 vars in 1147 bytes} [Sat Aug 27 02:35:31 2022] POST /compute/v2.1/servers => generated 402 bytes in 1800 msecs (HTTP/1.1 202) 10 headers in 451 bytes (2 switches on core 0) Aug 27 02:35:33.705631 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-d36cf8f8-e117-4511-8c4d-879da8f0d4a6 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:35:33.706838 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-d36cf8f8-e117-4511-8c4d-879da8f0d4a6 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] [instance: f5bad70c-350c-4230-a862-626dd6805e4c] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:35:33.810936 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-d36cf8f8-e117-4511-8c4d-879da8f0d4a6 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee6084fffc544dfe8f1a9a30ae06be69', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a8055f4326f640fc8927f5cfdef39dd3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:35:33.820039 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-d36cf8f8-e117-4511-8c4d-879da8f0d4a6 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] 10.222.0.52 "GET /compute/v2.1/servers/f5bad70c-350c-4230-a862-626dd6805e4c" status: 200 len: 1237 microversion: 2.1 time: 0.119186 Aug 27 02:35:33.820735 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 517/1040] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:35:33 2022] GET /compute/v2.1/servers/f5bad70c-350c-4230-a862-626dd6805e4c => generated 1237 bytes in 121 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 27 02:35:34.842699 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-3f2eec03-3cbf-413d-80df-3007c13e7931 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Calling method '>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:35:34.843680 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-3f2eec03-3cbf-413d-80df-3007c13e7931 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] [instance: f5bad70c-350c-4230-a862-626dd6805e4c] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:35:34.867564 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-3f2eec03-3cbf-413d-80df-3007c13e7931 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:35:34.868407 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-3f2eec03-3cbf-413d-80df-3007c13e7931 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:35:34.869229 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-3f2eec03-3cbf-413d-80df-3007c13e7931 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:35:34.871314 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-3f2eec03-3cbf-413d-80df-3007c13e7931 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:35:34.873406 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-3f2eec03-3cbf-413d-80df-3007c13e7931 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:35:34.878639 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-3f2eec03-3cbf-413d-80df-3007c13e7931 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:35:35.000765 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-3f2eec03-3cbf-413d-80df-3007c13e7931 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee6084fffc544dfe8f1a9a30ae06be69', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a8055f4326f640fc8927f5cfdef39dd3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:35:35.013226 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-3f2eec03-3cbf-413d-80df-3007c13e7931 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] 10.222.0.52 "GET /compute/v2.1/servers/f5bad70c-350c-4230-a862-626dd6805e4c" status: 200 len: 1289 microversion: 2.1 time: 0.172459 Aug 27 02:35:35.015638 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 524/1041] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:35:34 2022] GET /compute/v2.1/servers/f5bad70c-350c-4230-a862-626dd6805e4c => generated 1289 bytes in 179 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:35:36.037986 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-b8081472-a98c-40e3-9a01-0552172e42c6 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:35:36.038985 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-b8081472-a98c-40e3-9a01-0552172e42c6 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] [instance: f5bad70c-350c-4230-a862-626dd6805e4c] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:35:36.055527 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-b8081472-a98c-40e3-9a01-0552172e42c6 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:35:36.056044 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-b8081472-a98c-40e3-9a01-0552172e42c6 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:35:36.056487 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-b8081472-a98c-40e3-9a01-0552172e42c6 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:35:36.057270 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-b8081472-a98c-40e3-9a01-0552172e42c6 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:35:36.057733 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-b8081472-a98c-40e3-9a01-0552172e42c6 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:35:36.058259 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-b8081472-a98c-40e3-9a01-0552172e42c6 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:35:36.525301 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-b8081472-a98c-40e3-9a01-0552172e42c6 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee6084fffc544dfe8f1a9a30ae06be69', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a8055f4326f640fc8927f5cfdef39dd3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:35:36.536595 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-b8081472-a98c-40e3-9a01-0552172e42c6 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] 10.222.0.52 "GET /compute/v2.1/servers/f5bad70c-350c-4230-a862-626dd6805e4c" status: 200 len: 1295 microversion: 2.1 time: 0.503953 Aug 27 02:35:36.537701 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 518/1042] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:35:36 2022] GET /compute/v2.1/servers/f5bad70c-350c-4230-a862-626dd6805e4c => generated 1295 bytes in 506 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 27 02:35:37.562200 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-862a6ef2-8c56-49da-bc03-1bea2bf523f6 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Calling method '>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:35:37.564344 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-862a6ef2-8c56-49da-bc03-1bea2bf523f6 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] [instance: f5bad70c-350c-4230-a862-626dd6805e4c] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:35:37.587060 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-862a6ef2-8c56-49da-bc03-1bea2bf523f6 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:35:37.588116 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-862a6ef2-8c56-49da-bc03-1bea2bf523f6 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:35:37.588828 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-862a6ef2-8c56-49da-bc03-1bea2bf523f6 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:35:37.589945 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-862a6ef2-8c56-49da-bc03-1bea2bf523f6 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:35:37.590659 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-862a6ef2-8c56-49da-bc03-1bea2bf523f6 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:35:37.591254 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-862a6ef2-8c56-49da-bc03-1bea2bf523f6 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:35:37.971013 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-862a6ef2-8c56-49da-bc03-1bea2bf523f6 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee6084fffc544dfe8f1a9a30ae06be69', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a8055f4326f640fc8927f5cfdef39dd3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:35:37.992321 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-862a6ef2-8c56-49da-bc03-1bea2bf523f6 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] 10.222.0.52 "GET /compute/v2.1/servers/f5bad70c-350c-4230-a862-626dd6805e4c" status: 200 len: 1363 microversion: 2.1 time: 0.435498 Aug 27 02:35:38.003451 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 525/1043] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:35:37 2022] GET /compute/v2.1/servers/f5bad70c-350c-4230-a862-626dd6805e4c => generated 1363 bytes in 448 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:35:38.590964 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [req-43a82d82-a0a0-4f4b-9dba-bafe2c265018 req-26863d7e-0186-41a7-9143-eba31ec6e128 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "f5bad70c-350c-4230-a862-626dd6805e4c", "tag": "4e23603f-f7cd-42a0-9751-b3014224edb8"}]} {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 27 02:35:38.602503 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [req-43a82d82-a0a0-4f4b-9dba-bafe2c265018 req-26863d7e-0186-41a7-9143-eba31ec6e128 service nova] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:35:38.603289 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [req-43a82d82-a0a0-4f4b-9dba-bafe2c265018 req-26863d7e-0186-41a7-9143-eba31ec6e128 service nova] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:35:38.603924 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [req-43a82d82-a0a0-4f4b-9dba-bafe2c265018 req-26863d7e-0186-41a7-9143-eba31ec6e128 service nova] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:35:38.626810 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.compute.server_external_events [req-43a82d82-a0a0-4f4b-9dba-bafe2c265018 req-26863d7e-0186-41a7-9143-eba31ec6e128 service nova] Creating event network-changed:4e23603f-f7cd-42a0-9751-b3014224edb8 for instance f5bad70c-350c-4230-a862-626dd6805e4c on hv-cinder-84630 Aug 27 02:35:38.636746 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [req-43a82d82-a0a0-4f4b-9dba-bafe2c265018 req-26863d7e-0186-41a7-9143-eba31ec6e128 service nova] 10.222.0.52 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.049463 Aug 27 02:35:38.637985 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 519/1044] 10.222.0.52 () {64 vars in 1317 bytes} [Sat Aug 27 02:35:38 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 51 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 27 02:35:39.024644 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-c42d1173-b0b0-47af-ac7f-f1791cfc5adc tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:35:39.025594 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-c42d1173-b0b0-47af-ac7f-f1791cfc5adc tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] [instance: f5bad70c-350c-4230-a862-626dd6805e4c] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:35:39.038819 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-c42d1173-b0b0-47af-ac7f-f1791cfc5adc tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:35:39.039373 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-c42d1173-b0b0-47af-ac7f-f1791cfc5adc tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:35:39.040147 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-c42d1173-b0b0-47af-ac7f-f1791cfc5adc tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:35:39.041380 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-c42d1173-b0b0-47af-ac7f-f1791cfc5adc tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:35:39.042044 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-c42d1173-b0b0-47af-ac7f-f1791cfc5adc tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:35:39.042652 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-c42d1173-b0b0-47af-ac7f-f1791cfc5adc tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:35:39.277329 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-e9d6f854-afe2-4c64-9ce0-7dad941f7cba tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Action: 'create', calling method: .version_select at 0x7f6b1e7db8b0>, body: {"keypair": {"name": "tempest-TestVolumeBootPattern-1366561407", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBOgVYLZBJOLn+P4XJWUS20EUQ+T5hh7z/b3w0RCVBQf69O0bdQyW3lb8aOkmbdRWY/2emKab+EJcQ7XMoDp7XP3lw9dtHm50yLomccxDuiCP9SwV5GWQoRL5EEgJcuA6Dg=="}} {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 27 02:35:39.296232 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.quota [None req-e9d6f854-afe2-4c64-9ce0-7dad941f7cba tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Getting quotas for project 79e671941c234f36b542080d256de2c6. Resources: {'key_pairs'} {{(pid=120864) _get_quotas /opt/stack/nova/nova/quota.py:393}} Aug 27 02:35:39.300959 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.quota [None req-e9d6f854-afe2-4c64-9ce0-7dad941f7cba tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Getting quotas for user 3fe5a1a037fa460eb3121e4d307542da and project 79e671941c234f36b542080d256de2c6. Resources: {'key_pairs'} {{(pid=120864) _get_quotas /opt/stack/nova/nova/quota.py:383}} Aug 27 02:35:39.323826 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-c42d1173-b0b0-47af-ac7f-f1791cfc5adc tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee6084fffc544dfe8f1a9a30ae06be69', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a8055f4326f640fc8927f5cfdef39dd3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:35:39.330389 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-e9d6f854-afe2-4c64-9ce0-7dad941f7cba tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] 10.222.0.52 "POST /compute/v2.1/os-keypairs" status: 200 len: 400 microversion: 2.1 time: 0.357055 Aug 27 02:35:39.335549 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-c42d1173-b0b0-47af-ac7f-f1791cfc5adc tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] 10.222.0.52 "GET /compute/v2.1/servers/f5bad70c-350c-4230-a862-626dd6805e4c" status: 200 len: 1363 microversion: 2.1 time: 0.322859 Aug 27 02:35:39.336933 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 520/1045] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:35:39 2022] GET /compute/v2.1/servers/f5bad70c-350c-4230-a862-626dd6805e4c => generated 1363 bytes in 325 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:35:39.337634 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 526/1046] 10.222.0.52 () {60 vars in 1159 bytes} [Sat Aug 27 02:35:38 2022] POST /compute/v2.1/os-keypairs => generated 400 bytes in 365 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 27 02:35:40.358480 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-2006424b-38f0-4440-ad20-13990e72d8b5 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:35:40.359096 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-2006424b-38f0-4440-ad20-13990e72d8b5 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] [instance: f5bad70c-350c-4230-a862-626dd6805e4c] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:35:40.366440 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-2006424b-38f0-4440-ad20-13990e72d8b5 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:35:40.366930 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-2006424b-38f0-4440-ad20-13990e72d8b5 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:35:40.367368 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-2006424b-38f0-4440-ad20-13990e72d8b5 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:35:40.368074 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-2006424b-38f0-4440-ad20-13990e72d8b5 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:35:40.368522 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-2006424b-38f0-4440-ad20-13990e72d8b5 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:35:40.368943 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-2006424b-38f0-4440-ad20-13990e72d8b5 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:35:40.530635 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-2006424b-38f0-4440-ad20-13990e72d8b5 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee6084fffc544dfe8f1a9a30ae06be69', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a8055f4326f640fc8927f5cfdef39dd3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:35:40.542071 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-2006424b-38f0-4440-ad20-13990e72d8b5 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] 10.222.0.52 "GET /compute/v2.1/servers/f5bad70c-350c-4230-a862-626dd6805e4c" status: 200 len: 1363 microversion: 2.1 time: 0.187862 Aug 27 02:35:40.543043 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 521/1047] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:35:40 2022] GET /compute/v2.1/servers/f5bad70c-350c-4230-a862-626dd6805e4c => generated 1363 bytes in 190 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:35:41.570059 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-fb2dafd2-eb27-43c0-8271-f6488326f0c0 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Calling method '>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:35:41.571401 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-fb2dafd2-eb27-43c0-8271-f6488326f0c0 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] [instance: f5bad70c-350c-4230-a862-626dd6805e4c] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:35:41.597208 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-fb2dafd2-eb27-43c0-8271-f6488326f0c0 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:35:41.603465 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-fb2dafd2-eb27-43c0-8271-f6488326f0c0 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.006s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:35:41.604251 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-fb2dafd2-eb27-43c0-8271-f6488326f0c0 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:35:41.607924 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-fb2dafd2-eb27-43c0-8271-f6488326f0c0 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:35:41.609292 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-fb2dafd2-eb27-43c0-8271-f6488326f0c0 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:35:41.610532 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-fb2dafd2-eb27-43c0-8271-f6488326f0c0 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:35:41.866493 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-fb2dafd2-eb27-43c0-8271-f6488326f0c0 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee6084fffc544dfe8f1a9a30ae06be69', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a8055f4326f640fc8927f5cfdef39dd3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:35:41.875836 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-fb2dafd2-eb27-43c0-8271-f6488326f0c0 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] 10.222.0.52 "GET /compute/v2.1/servers/f5bad70c-350c-4230-a862-626dd6805e4c" status: 200 len: 1363 microversion: 2.1 time: 0.312119 Aug 27 02:35:41.876424 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 527/1048] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:35:41 2022] GET /compute/v2.1/servers/f5bad70c-350c-4230-a862-626dd6805e4c => generated 1363 bytes in 314 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:35:42.907629 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-bcab1d15-e19f-4dc6-aa82-e8a216bb6619 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:35:42.912145 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-bcab1d15-e19f-4dc6-aa82-e8a216bb6619 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] [instance: f5bad70c-350c-4230-a862-626dd6805e4c] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:35:42.924507 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-bcab1d15-e19f-4dc6-aa82-e8a216bb6619 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:35:42.925007 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-bcab1d15-e19f-4dc6-aa82-e8a216bb6619 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:35:42.925673 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-bcab1d15-e19f-4dc6-aa82-e8a216bb6619 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:35:42.927433 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-bcab1d15-e19f-4dc6-aa82-e8a216bb6619 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:35:42.927853 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-bcab1d15-e19f-4dc6-aa82-e8a216bb6619 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:35:42.928340 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-bcab1d15-e19f-4dc6-aa82-e8a216bb6619 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:35:43.187096 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-bcab1d15-e19f-4dc6-aa82-e8a216bb6619 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee6084fffc544dfe8f1a9a30ae06be69', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a8055f4326f640fc8927f5cfdef39dd3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:35:43.199203 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-bcab1d15-e19f-4dc6-aa82-e8a216bb6619 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] 10.222.0.52 "GET /compute/v2.1/servers/f5bad70c-350c-4230-a862-626dd6805e4c" status: 200 len: 1363 microversion: 2.1 time: 0.294256 Aug 27 02:35:43.199544 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 522/1049] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:35:42 2022] GET /compute/v2.1/servers/f5bad70c-350c-4230-a862-626dd6805e4c => generated 1363 bytes in 299 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:35:44.228595 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-fc3ea2c3-7bdb-4b02-9caf-fc557e3a5515 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Calling method '>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:35:44.229323 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-fc3ea2c3-7bdb-4b02-9caf-fc557e3a5515 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] [instance: f5bad70c-350c-4230-a862-626dd6805e4c] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:35:44.232257 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG dbcounter [-] [120864] Writing DB stats nova_cell0:SELECT=2 {{(pid=120864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 27 02:35:44.241595 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-fc3ea2c3-7bdb-4b02-9caf-fc557e3a5515 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:35:44.242214 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-fc3ea2c3-7bdb-4b02-9caf-fc557e3a5515 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:35:44.242698 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-fc3ea2c3-7bdb-4b02-9caf-fc557e3a5515 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:35:44.243658 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-fc3ea2c3-7bdb-4b02-9caf-fc557e3a5515 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:35:44.244154 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-fc3ea2c3-7bdb-4b02-9caf-fc557e3a5515 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:35:44.244837 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-fc3ea2c3-7bdb-4b02-9caf-fc557e3a5515 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:35:44.395474 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-fc3ea2c3-7bdb-4b02-9caf-fc557e3a5515 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee6084fffc544dfe8f1a9a30ae06be69', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a8055f4326f640fc8927f5cfdef39dd3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:35:44.404917 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-fc3ea2c3-7bdb-4b02-9caf-fc557e3a5515 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] 10.222.0.52 "GET /compute/v2.1/servers/f5bad70c-350c-4230-a862-626dd6805e4c" status: 200 len: 1363 microversion: 2.1 time: 0.180727 Aug 27 02:35:44.405546 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 528/1050] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:35:44 2022] GET /compute/v2.1/servers/f5bad70c-350c-4230-a862-626dd6805e4c => generated 1363 bytes in 182 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 27 02:35:45.031512 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [req-cdce3b2a-61a4-423d-80e3-fece713bfd59 req-578786fe-83f3-4af8-a0cb-20b3fdb16f60 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "f5bad70c-350c-4230-a862-626dd6805e4c", "name": "network-vif-plugged", "status": "completed", "tag": "4e23603f-f7cd-42a0-9751-b3014224edb8"}]} {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 27 02:35:45.038403 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG dbcounter [-] [120863] Writing DB stats nova_cell0:SELECT=6 {{(pid=120863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 27 02:35:45.047013 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [req-cdce3b2a-61a4-423d-80e3-fece713bfd59 req-578786fe-83f3-4af8-a0cb-20b3fdb16f60 service nova] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:35:45.047751 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [req-cdce3b2a-61a4-423d-80e3-fece713bfd59 req-578786fe-83f3-4af8-a0cb-20b3fdb16f60 service nova] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:35:45.048282 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [req-cdce3b2a-61a4-423d-80e3-fece713bfd59 req-578786fe-83f3-4af8-a0cb-20b3fdb16f60 service nova] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:35:45.063692 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.compute.server_external_events [req-cdce3b2a-61a4-423d-80e3-fece713bfd59 req-578786fe-83f3-4af8-a0cb-20b3fdb16f60 service nova] Creating event network-vif-plugged:4e23603f-f7cd-42a0-9751-b3014224edb8 for instance f5bad70c-350c-4230-a862-626dd6805e4c on hv-cinder-84630 Aug 27 02:35:45.072671 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [req-cdce3b2a-61a4-423d-80e3-fece713bfd59 req-578786fe-83f3-4af8-a0cb-20b3fdb16f60 service nova] 10.222.0.52 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.044891 Aug 27 02:35:45.073638 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 523/1051] 10.222.0.52 () {64 vars in 1317 bytes} [Sat Aug 27 02:35:45 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 27 02:35:45.428308 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-be2b5654-edee-422a-8ad9-9066884f5be0 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Calling method '>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:35:45.428909 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-be2b5654-edee-422a-8ad9-9066884f5be0 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] [instance: f5bad70c-350c-4230-a862-626dd6805e4c] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:35:45.437490 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-be2b5654-edee-422a-8ad9-9066884f5be0 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:35:45.438220 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-be2b5654-edee-422a-8ad9-9066884f5be0 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:35:45.438802 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-be2b5654-edee-422a-8ad9-9066884f5be0 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:35:45.439716 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-be2b5654-edee-422a-8ad9-9066884f5be0 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:35:45.440264 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-be2b5654-edee-422a-8ad9-9066884f5be0 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:35:45.440865 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-be2b5654-edee-422a-8ad9-9066884f5be0 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:35:45.591052 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-be2b5654-edee-422a-8ad9-9066884f5be0 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee6084fffc544dfe8f1a9a30ae06be69', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a8055f4326f640fc8927f5cfdef39dd3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:35:45.610339 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-be2b5654-edee-422a-8ad9-9066884f5be0 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] 10.222.0.52 "GET /compute/v2.1/servers/f5bad70c-350c-4230-a862-626dd6805e4c" status: 200 len: 1363 microversion: 2.1 time: 0.186293 Aug 27 02:35:45.610970 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 529/1052] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:35:45 2022] GET /compute/v2.1/servers/f5bad70c-350c-4230-a862-626dd6805e4c => generated 1363 bytes in 188 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:35:46.633093 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-5b1d29db-a129-449e-be37-0ebd71964a48 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:35:46.633866 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-5b1d29db-a129-449e-be37-0ebd71964a48 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] [instance: f5bad70c-350c-4230-a862-626dd6805e4c] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:35:46.641157 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-5b1d29db-a129-449e-be37-0ebd71964a48 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:35:46.641683 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-5b1d29db-a129-449e-be37-0ebd71964a48 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:35:46.642141 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-5b1d29db-a129-449e-be37-0ebd71964a48 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:35:46.642979 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-5b1d29db-a129-449e-be37-0ebd71964a48 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:35:46.643492 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-5b1d29db-a129-449e-be37-0ebd71964a48 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:35:46.643911 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-5b1d29db-a129-449e-be37-0ebd71964a48 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:35:46.800832 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-5b1d29db-a129-449e-be37-0ebd71964a48 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee6084fffc544dfe8f1a9a30ae06be69', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a8055f4326f640fc8927f5cfdef39dd3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:35:46.809485 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-5b1d29db-a129-449e-be37-0ebd71964a48 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] 10.222.0.52 "GET /compute/v2.1/servers/f5bad70c-350c-4230-a862-626dd6805e4c" status: 200 len: 1363 microversion: 2.1 time: 0.180910 Aug 27 02:35:46.810190 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 524/1053] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:35:46 2022] GET /compute/v2.1/servers/f5bad70c-350c-4230-a862-626dd6805e4c => generated 1363 bytes in 183 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:35:47.834267 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-faf32362-cb33-4227-993d-eb63a96517c0 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Calling method '>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:35:47.834950 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-faf32362-cb33-4227-993d-eb63a96517c0 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] [instance: f5bad70c-350c-4230-a862-626dd6805e4c] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:35:47.842430 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-faf32362-cb33-4227-993d-eb63a96517c0 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:35:47.842933 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-faf32362-cb33-4227-993d-eb63a96517c0 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:35:47.843365 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-faf32362-cb33-4227-993d-eb63a96517c0 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:35:47.844070 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-faf32362-cb33-4227-993d-eb63a96517c0 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:35:47.844517 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-faf32362-cb33-4227-993d-eb63a96517c0 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:35:47.844922 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-faf32362-cb33-4227-993d-eb63a96517c0 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:35:47.984374 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-faf32362-cb33-4227-993d-eb63a96517c0 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee6084fffc544dfe8f1a9a30ae06be69', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a8055f4326f640fc8927f5cfdef39dd3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:35:47.992604 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-faf32362-cb33-4227-993d-eb63a96517c0 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] 10.222.0.52 "GET /compute/v2.1/servers/f5bad70c-350c-4230-a862-626dd6805e4c" status: 200 len: 1548 microversion: 2.1 time: 0.162267 Aug 27 02:35:47.993292 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 530/1054] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:35:47 2022] GET /compute/v2.1/servers/f5bad70c-350c-4230-a862-626dd6805e4c => generated 1548 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:35:48.014527 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-1a2a19bf-3474-49c4-95af-0d524c0c5a40 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:35:48.015122 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-1a2a19bf-3474-49c4-95af-0d524c0c5a40 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] [instance: f5bad70c-350c-4230-a862-626dd6805e4c] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:35:48.021668 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-1a2a19bf-3474-49c4-95af-0d524c0c5a40 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:35:48.022155 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-1a2a19bf-3474-49c4-95af-0d524c0c5a40 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:35:48.022584 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-1a2a19bf-3474-49c4-95af-0d524c0c5a40 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:35:48.023655 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-1a2a19bf-3474-49c4-95af-0d524c0c5a40 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:35:48.024161 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-1a2a19bf-3474-49c4-95af-0d524c0c5a40 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:35:48.024581 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-1a2a19bf-3474-49c4-95af-0d524c0c5a40 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:35:48.109738 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-1a2a19bf-3474-49c4-95af-0d524c0c5a40 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] 10.222.0.52 "GET /compute/v2.1/servers/f5bad70c-350c-4230-a862-626dd6805e4c/os-interface" status: 200 len: 284 microversion: 2.1 time: 0.099368 Aug 27 02:35:48.110335 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 525/1055] 10.222.0.52 () {58 vars in 1275 bytes} [Sat Aug 27 02:35:48 2022] GET /compute/v2.1/servers/f5bad70c-350c-4230-a862-626dd6805e4c/os-interface => generated 284 bytes in 101 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 27 02:35:49.261392 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [req-6c053e43-6090-4707-b74b-c92ec0bf5603 req-e1b70b2a-1345-4d32-b0dd-dad678e6a944 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "f5bad70c-350c-4230-a862-626dd6805e4c", "tag": "4e23603f-f7cd-42a0-9751-b3014224edb8"}]} {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 27 02:35:49.274082 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [req-6c053e43-6090-4707-b74b-c92ec0bf5603 req-e1b70b2a-1345-4d32-b0dd-dad678e6a944 service nova] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:35:49.274843 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [req-6c053e43-6090-4707-b74b-c92ec0bf5603 req-e1b70b2a-1345-4d32-b0dd-dad678e6a944 service nova] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:35:49.275497 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [req-6c053e43-6090-4707-b74b-c92ec0bf5603 req-e1b70b2a-1345-4d32-b0dd-dad678e6a944 service nova] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:35:49.287811 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.compute.server_external_events [req-6c053e43-6090-4707-b74b-c92ec0bf5603 req-e1b70b2a-1345-4d32-b0dd-dad678e6a944 service nova] Creating event network-changed:4e23603f-f7cd-42a0-9751-b3014224edb8 for instance f5bad70c-350c-4230-a862-626dd6805e4c on hv-cinder-84630 Aug 27 02:35:49.293953 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [req-6c053e43-6090-4707-b74b-c92ec0bf5603 req-e1b70b2a-1345-4d32-b0dd-dad678e6a944 service nova] 10.222.0.52 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.101484 Aug 27 02:35:49.294451 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 531/1056] 10.222.0.52 () {64 vars in 1317 bytes} [Sat Aug 27 02:35:49 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 103 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 27 02:36:03.395661 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG dbcounter [-] [120863] Writing DB stats nova_api:SELECT=31,nova_api:INSERT=1 {{(pid=120863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 27 02:36:03.396512 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG dbcounter [-] [120863] Writing DB stats nova_cell1:SELECT=38 {{(pid=120863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 27 02:36:03.400777 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-c7c805a2-00b3-41d3-aeea-9e82b67e5093 tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-TestVolumeBootPattern-server-1061509144", "imageRef": "", "flavorRef": "452", "key_name": "tempest-TestVolumeBootPattern-1366561407", "security_groups": [{"name": "tempest-secgroup-smoke-743539963"}], "block_device_mapping_v2": [{"uuid": "8509b4bf-c8cd-41be-817e-d010c8fe3bdd", "source_type": "volume", "destination_type": "volume", "boot_index": 0, "delete_on_termination": false}], "networks": [{"uuid": "3a1ac093-e4b6-48ea-aff8-2575f231ba35"}]}} {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 27 02:36:03.522148 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.network.neutron [None req-c7c805a2-00b3-41d3-aeea-9e82b67e5093 tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] validate_networks() for [('3a1ac093-e4b6-48ea-aff8-2575f231ba35', None, None, None, None, None)] {{(pid=120863) validate_networks /opt/stack/nova/nova/network/neutron.py:2626}} Aug 27 02:36:04.310494 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.virt.hardware [None req-c7c805a2-00b3-41d3-aeea-9e82b67e5093 tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Flavor limits 0:0:0 {{(pid=120863) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Aug 27 02:36:04.311176 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.virt.hardware [None req-c7c805a2-00b3-41d3-aeea-9e82b67e5093 tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Image limits 0:0:0 {{(pid=120863) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Aug 27 02:36:04.311977 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.virt.hardware [None req-c7c805a2-00b3-41d3-aeea-9e82b67e5093 tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Flavor pref 0:0:0 {{(pid=120863) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Aug 27 02:36:04.312606 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.virt.hardware [None req-c7c805a2-00b3-41d3-aeea-9e82b67e5093 tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Image pref 0:0:0 {{(pid=120863) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Aug 27 02:36:04.313500 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.virt.hardware [None req-c7c805a2-00b3-41d3-aeea-9e82b67e5093 tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=120863) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Aug 27 02:36:04.325876 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-c7c805a2-00b3-41d3-aeea-9e82b67e5093 tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:36:04.326367 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-c7c805a2-00b3-41d3-aeea-9e82b67e5093 tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:36:04.326869 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-c7c805a2-00b3-41d3-aeea-9e82b67e5093 tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:36:04.327594 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-c7c805a2-00b3-41d3-aeea-9e82b67e5093 tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:36:04.328046 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-c7c805a2-00b3-41d3-aeea-9e82b67e5093 tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:36:04.328512 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-c7c805a2-00b3-41d3-aeea-9e82b67e5093 tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:36:04.346436 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.quota [None req-c7c805a2-00b3-41d3-aeea-9e82b67e5093 tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Getting quotas for project 79e671941c234f36b542080d256de2c6. Resources: {'cores', 'ram', 'instances'} {{(pid=120863) _get_quotas /opt/stack/nova/nova/quota.py:393}} Aug 27 02:36:04.354138 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.quota [None req-c7c805a2-00b3-41d3-aeea-9e82b67e5093 tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Getting quotas for user 3fe5a1a037fa460eb3121e4d307542da and project 79e671941c234f36b542080d256de2c6. Resources: {'cores', 'ram', 'instances'} {{(pid=120863) _get_quotas /opt/stack/nova/nova/quota.py:383}} Aug 27 02:36:04.435078 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-c7c805a2-00b3-41d3-aeea-9e82b67e5093 tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Going to run 1 instances... {{(pid=120863) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Aug 27 02:36:04.438447 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-c7c805a2-00b3-41d3-aeea-9e82b67e5093 tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] [instance: 30aa22c4-59b4-4482-b92a-3f386466f7ee] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=False,deleted=,deleted_at=,destination_type='volume',device_name=None,device_type=None,disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id=None,instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='volume',tag=None,updated_at=,uuid=,volume_id='8509b4bf-c8cd-41be-817e-d010c8fe3bdd',volume_size=None,volume_type=None)] {{(pid=120863) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Aug 27 02:36:04.617837 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-c7c805a2-00b3-41d3-aeea-9e82b67e5093 tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] 10.222.0.52 "POST /compute/v2.1/servers" status: 202 len: 405 microversion: 2.1 time: 1.223419 Aug 27 02:36:04.617837 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 526/1057] 10.222.0.52 () {60 vars in 1147 bytes} [Sat Aug 27 02:36:03 2022] POST /compute/v2.1/servers => generated 405 bytes in 1225 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Aug 27 02:36:04.628630 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG dbcounter [-] [120864] Writing DB stats nova_cell0:SELECT=10 {{(pid=120864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 27 02:36:04.630717 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG dbcounter [-] [120864] Writing DB stats nova_api:SELECT=40,nova_api:INSERT=4 {{(pid=120864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 27 02:36:04.631417 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG dbcounter [-] [120864] Writing DB stats nova_cell1:SELECT=40 {{(pid=120864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 27 02:36:04.637006 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-96000241-08fb-428c-b17c-98002ff0c861 tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Calling method '>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:36:04.637895 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-96000241-08fb-428c-b17c-98002ff0c861 tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] [instance: 30aa22c4-59b4-4482-b92a-3f386466f7ee] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:36:04.710624 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-96000241-08fb-428c-b17c-98002ff0c861 tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3fe5a1a037fa460eb3121e4d307542da', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '79e671941c234f36b542080d256de2c6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:36:04.718673 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-96000241-08fb-428c-b17c-98002ff0c861 tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] 10.222.0.52 "GET /compute/v2.1/servers/30aa22c4-59b4-4482-b92a-3f386466f7ee" status: 200 len: 1095 microversion: 2.1 time: 0.090887 Aug 27 02:36:04.719342 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 532/1058] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:36:04 2022] GET /compute/v2.1/servers/30aa22c4-59b4-4482-b92a-3f386466f7ee => generated 1095 bytes in 93 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:36:05.741617 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-f4e742e8-f8ad-4ff9-84c4-43269d81fbde tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:36:05.742226 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-f4e742e8-f8ad-4ff9-84c4-43269d81fbde tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] [instance: 30aa22c4-59b4-4482-b92a-3f386466f7ee] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:36:05.750276 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-f4e742e8-f8ad-4ff9-84c4-43269d81fbde tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:36:05.750767 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-f4e742e8-f8ad-4ff9-84c4-43269d81fbde tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:36:05.751204 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-f4e742e8-f8ad-4ff9-84c4-43269d81fbde tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:36:05.751907 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-f4e742e8-f8ad-4ff9-84c4-43269d81fbde tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:36:05.752367 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-f4e742e8-f8ad-4ff9-84c4-43269d81fbde tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:36:05.752791 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-f4e742e8-f8ad-4ff9-84c4-43269d81fbde tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:36:05.818032 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-f4e742e8-f8ad-4ff9-84c4-43269d81fbde tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3fe5a1a037fa460eb3121e4d307542da', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '79e671941c234f36b542080d256de2c6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:36:05.826533 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-f4e742e8-f8ad-4ff9-84c4-43269d81fbde tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] 10.222.0.52 "GET /compute/v2.1/servers/30aa22c4-59b4-4482-b92a-3f386466f7ee" status: 200 len: 1193 microversion: 2.1 time: 0.089947 Aug 27 02:36:05.827136 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 527/1059] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:36:05 2022] GET /compute/v2.1/servers/30aa22c4-59b4-4482-b92a-3f386466f7ee => generated 1193 bytes in 91 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:36:06.851991 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-969792ea-1e76-4527-9d6c-5bf676b0dcca tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Calling method '>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:36:06.852822 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-969792ea-1e76-4527-9d6c-5bf676b0dcca tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] [instance: 30aa22c4-59b4-4482-b92a-3f386466f7ee] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:36:06.870422 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-969792ea-1e76-4527-9d6c-5bf676b0dcca tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:36:06.871012 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-969792ea-1e76-4527-9d6c-5bf676b0dcca tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:36:06.871462 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-969792ea-1e76-4527-9d6c-5bf676b0dcca tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:36:06.872379 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-969792ea-1e76-4527-9d6c-5bf676b0dcca tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:36:06.872850 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-969792ea-1e76-4527-9d6c-5bf676b0dcca tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:36:06.873367 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-969792ea-1e76-4527-9d6c-5bf676b0dcca tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:36:07.150144 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-969792ea-1e76-4527-9d6c-5bf676b0dcca tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3fe5a1a037fa460eb3121e4d307542da', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '79e671941c234f36b542080d256de2c6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:36:07.158463 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-969792ea-1e76-4527-9d6c-5bf676b0dcca tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] 10.222.0.52 "GET /compute/v2.1/servers/30aa22c4-59b4-4482-b92a-3f386466f7ee" status: 200 len: 1278 microversion: 2.1 time: 0.312461 Aug 27 02:36:07.159144 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 533/1060] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:36:06 2022] GET /compute/v2.1/servers/30aa22c4-59b4-4482-b92a-3f386466f7ee => generated 1278 bytes in 315 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:36:07.567533 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [req-0d50e67d-83d8-458e-a16f-fe688c9320e7 req-8ef303d9-6560-4eac-a26c-cc3f3b86c0e3 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "30aa22c4-59b4-4482-b92a-3f386466f7ee", "tag": "d3b3018c-6150-428a-a235-9ccd41ba80e7"}]} {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 27 02:36:07.579762 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [req-0d50e67d-83d8-458e-a16f-fe688c9320e7 req-8ef303d9-6560-4eac-a26c-cc3f3b86c0e3 service nova] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:36:07.580574 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [req-0d50e67d-83d8-458e-a16f-fe688c9320e7 req-8ef303d9-6560-4eac-a26c-cc3f3b86c0e3 service nova] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:36:07.581185 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [req-0d50e67d-83d8-458e-a16f-fe688c9320e7 req-8ef303d9-6560-4eac-a26c-cc3f3b86c0e3 service nova] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:36:07.594271 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.compute.server_external_events [req-0d50e67d-83d8-458e-a16f-fe688c9320e7 req-8ef303d9-6560-4eac-a26c-cc3f3b86c0e3 service nova] Creating event network-changed:d3b3018c-6150-428a-a235-9ccd41ba80e7 for instance 30aa22c4-59b4-4482-b92a-3f386466f7ee on hv-cinder-84630 Aug 27 02:36:07.599750 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [req-0d50e67d-83d8-458e-a16f-fe688c9320e7 req-8ef303d9-6560-4eac-a26c-cc3f3b86c0e3 service nova] 10.222.0.52 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.036431 Aug 27 02:36:07.600446 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 528/1061] 10.222.0.52 () {64 vars in 1317 bytes} [Sat Aug 27 02:36:07 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 38 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 27 02:36:08.180593 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-a26e8822-46ef-48e2-a4b4-05bbd2b51054 tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Calling method '>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:36:08.181318 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-a26e8822-46ef-48e2-a4b4-05bbd2b51054 tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] [instance: 30aa22c4-59b4-4482-b92a-3f386466f7ee] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:36:08.187859 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-a26e8822-46ef-48e2-a4b4-05bbd2b51054 tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:36:08.188336 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-a26e8822-46ef-48e2-a4b4-05bbd2b51054 tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:36:08.188758 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-a26e8822-46ef-48e2-a4b4-05bbd2b51054 tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:36:08.189490 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-a26e8822-46ef-48e2-a4b4-05bbd2b51054 tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:36:08.189933 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-a26e8822-46ef-48e2-a4b4-05bbd2b51054 tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:36:08.190344 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-a26e8822-46ef-48e2-a4b4-05bbd2b51054 tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:36:08.339938 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-a26e8822-46ef-48e2-a4b4-05bbd2b51054 tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3fe5a1a037fa460eb3121e4d307542da', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '79e671941c234f36b542080d256de2c6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:36:08.348017 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-a26e8822-46ef-48e2-a4b4-05bbd2b51054 tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] 10.222.0.52 "GET /compute/v2.1/servers/30aa22c4-59b4-4482-b92a-3f386466f7ee" status: 200 len: 1266 microversion: 2.1 time: 0.171647 Aug 27 02:36:08.348596 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 534/1062] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:36:08 2022] GET /compute/v2.1/servers/30aa22c4-59b4-4482-b92a-3f386466f7ee => generated 1266 bytes in 173 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:36:09.371168 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-1410e282-0f3e-411a-8c1a-cf972d3d7073 tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:36:09.371884 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-1410e282-0f3e-411a-8c1a-cf972d3d7073 tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] [instance: 30aa22c4-59b4-4482-b92a-3f386466f7ee] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:36:09.378585 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-1410e282-0f3e-411a-8c1a-cf972d3d7073 tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:36:09.379099 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-1410e282-0f3e-411a-8c1a-cf972d3d7073 tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:36:09.379534 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-1410e282-0f3e-411a-8c1a-cf972d3d7073 tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:36:09.380231 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-1410e282-0f3e-411a-8c1a-cf972d3d7073 tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:36:09.380763 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-1410e282-0f3e-411a-8c1a-cf972d3d7073 tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:36:09.381174 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-1410e282-0f3e-411a-8c1a-cf972d3d7073 tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:36:09.591916 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-1410e282-0f3e-411a-8c1a-cf972d3d7073 tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3fe5a1a037fa460eb3121e4d307542da', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '79e671941c234f36b542080d256de2c6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:36:09.601969 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-1410e282-0f3e-411a-8c1a-cf972d3d7073 tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] 10.222.0.52 "GET /compute/v2.1/servers/30aa22c4-59b4-4482-b92a-3f386466f7ee" status: 200 len: 1266 microversion: 2.1 time: 0.235744 Aug 27 02:36:09.602811 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 529/1063] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:36:09 2022] GET /compute/v2.1/servers/30aa22c4-59b4-4482-b92a-3f386466f7ee => generated 1266 bytes in 237 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:36:10.625351 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-f5f7747e-49dc-4cd6-a772-6d79a4164468 tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Calling method '>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:36:10.625985 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-f5f7747e-49dc-4cd6-a772-6d79a4164468 tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] [instance: 30aa22c4-59b4-4482-b92a-3f386466f7ee] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:36:10.633088 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-f5f7747e-49dc-4cd6-a772-6d79a4164468 tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:36:10.633836 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-f5f7747e-49dc-4cd6-a772-6d79a4164468 tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:36:10.634318 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-f5f7747e-49dc-4cd6-a772-6d79a4164468 tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:36:10.635073 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-f5f7747e-49dc-4cd6-a772-6d79a4164468 tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:36:10.635521 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-f5f7747e-49dc-4cd6-a772-6d79a4164468 tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:36:10.635938 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-f5f7747e-49dc-4cd6-a772-6d79a4164468 tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:36:10.793733 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-f5f7747e-49dc-4cd6-a772-6d79a4164468 tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3fe5a1a037fa460eb3121e4d307542da', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '79e671941c234f36b542080d256de2c6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:36:10.804822 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-f5f7747e-49dc-4cd6-a772-6d79a4164468 tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] 10.222.0.52 "GET /compute/v2.1/servers/30aa22c4-59b4-4482-b92a-3f386466f7ee" status: 200 len: 1266 microversion: 2.1 time: 0.184447 Aug 27 02:36:10.805795 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 535/1064] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:36:10 2022] GET /compute/v2.1/servers/30aa22c4-59b4-4482-b92a-3f386466f7ee => generated 1266 bytes in 186 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:36:11.840601 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-87158c75-58d9-428b-993b-05000fc03a92 tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:36:11.841542 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-87158c75-58d9-428b-993b-05000fc03a92 tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] [instance: 30aa22c4-59b4-4482-b92a-3f386466f7ee] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:36:11.850625 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-87158c75-58d9-428b-993b-05000fc03a92 tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:36:11.851611 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-87158c75-58d9-428b-993b-05000fc03a92 tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:36:11.852475 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-87158c75-58d9-428b-993b-05000fc03a92 tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:36:11.853863 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-87158c75-58d9-428b-993b-05000fc03a92 tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:36:11.854722 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-87158c75-58d9-428b-993b-05000fc03a92 tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:36:11.855420 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-87158c75-58d9-428b-993b-05000fc03a92 tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:36:12.017011 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-87158c75-58d9-428b-993b-05000fc03a92 tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3fe5a1a037fa460eb3121e4d307542da', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '79e671941c234f36b542080d256de2c6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:36:12.028533 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-87158c75-58d9-428b-993b-05000fc03a92 tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] 10.222.0.52 "GET /compute/v2.1/servers/30aa22c4-59b4-4482-b92a-3f386466f7ee" status: 200 len: 1266 microversion: 2.1 time: 0.194361 Aug 27 02:36:12.029685 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 530/1065] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:36:11 2022] GET /compute/v2.1/servers/30aa22c4-59b4-4482-b92a-3f386466f7ee => generated 1266 bytes in 197 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:36:13.056801 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-b013ebd2-db2f-4486-b441-65a064ad4368 tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Calling method '>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:36:13.057539 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-b013ebd2-db2f-4486-b441-65a064ad4368 tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] [instance: 30aa22c4-59b4-4482-b92a-3f386466f7ee] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:36:13.064117 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-b013ebd2-db2f-4486-b441-65a064ad4368 tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:36:13.064602 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-b013ebd2-db2f-4486-b441-65a064ad4368 tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:36:13.065037 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-b013ebd2-db2f-4486-b441-65a064ad4368 tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:36:13.065766 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-b013ebd2-db2f-4486-b441-65a064ad4368 tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:36:13.066216 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-b013ebd2-db2f-4486-b441-65a064ad4368 tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:36:13.066631 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-b013ebd2-db2f-4486-b441-65a064ad4368 tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:36:13.208559 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-b013ebd2-db2f-4486-b441-65a064ad4368 tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3fe5a1a037fa460eb3121e4d307542da', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '79e671941c234f36b542080d256de2c6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:36:13.219159 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-b013ebd2-db2f-4486-b441-65a064ad4368 tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] 10.222.0.52 "GET /compute/v2.1/servers/30aa22c4-59b4-4482-b92a-3f386466f7ee" status: 200 len: 1266 microversion: 2.1 time: 0.166407 Aug 27 02:36:13.219753 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 536/1066] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:36:13 2022] GET /compute/v2.1/servers/30aa22c4-59b4-4482-b92a-3f386466f7ee => generated 1266 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:36:14.243170 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-04e3096c-3715-4de7-ac11-2d1ca7210321 tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:36:14.244029 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-04e3096c-3715-4de7-ac11-2d1ca7210321 tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] [instance: 30aa22c4-59b4-4482-b92a-3f386466f7ee] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:36:14.251612 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-04e3096c-3715-4de7-ac11-2d1ca7210321 tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:36:14.252098 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-04e3096c-3715-4de7-ac11-2d1ca7210321 tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:36:14.252527 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-04e3096c-3715-4de7-ac11-2d1ca7210321 tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:36:14.253366 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-04e3096c-3715-4de7-ac11-2d1ca7210321 tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:36:14.253818 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-04e3096c-3715-4de7-ac11-2d1ca7210321 tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:36:14.254237 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-04e3096c-3715-4de7-ac11-2d1ca7210321 tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:36:14.335603 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG dbcounter [-] [120863] Writing DB stats nova_cell0:SELECT=2 {{(pid=120863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 27 02:36:14.362613 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG dbcounter [-] [120863] Writing DB stats nova_cell0:SELECT=6 {{(pid=120863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 27 02:36:14.393628 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-04e3096c-3715-4de7-ac11-2d1ca7210321 tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3fe5a1a037fa460eb3121e4d307542da', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '79e671941c234f36b542080d256de2c6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:36:14.403415 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-04e3096c-3715-4de7-ac11-2d1ca7210321 tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] 10.222.0.52 "GET /compute/v2.1/servers/30aa22c4-59b4-4482-b92a-3f386466f7ee" status: 200 len: 1266 microversion: 2.1 time: 0.164074 Aug 27 02:36:14.405981 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 531/1067] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:36:14 2022] GET /compute/v2.1/servers/30aa22c4-59b4-4482-b92a-3f386466f7ee => generated 1266 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:36:15.426188 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-5d2aceee-9140-4b49-8298-37108c5cac2c tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Calling method '>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:36:15.426769 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-5d2aceee-9140-4b49-8298-37108c5cac2c tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] [instance: 30aa22c4-59b4-4482-b92a-3f386466f7ee] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:36:15.429529 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG dbcounter [-] [120864] Writing DB stats nova_cell0:SELECT=2 {{(pid=120864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 27 02:36:15.435605 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-5d2aceee-9140-4b49-8298-37108c5cac2c tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:36:15.436082 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-5d2aceee-9140-4b49-8298-37108c5cac2c tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:36:15.436504 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-5d2aceee-9140-4b49-8298-37108c5cac2c tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:36:15.437231 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-5d2aceee-9140-4b49-8298-37108c5cac2c tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:36:15.437677 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-5d2aceee-9140-4b49-8298-37108c5cac2c tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:36:15.438350 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-5d2aceee-9140-4b49-8298-37108c5cac2c tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:36:15.581062 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-5d2aceee-9140-4b49-8298-37108c5cac2c tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3fe5a1a037fa460eb3121e4d307542da', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '79e671941c234f36b542080d256de2c6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:36:15.589761 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-5d2aceee-9140-4b49-8298-37108c5cac2c tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] 10.222.0.52 "GET /compute/v2.1/servers/30aa22c4-59b4-4482-b92a-3f386466f7ee" status: 200 len: 1266 microversion: 2.1 time: 0.167557 Aug 27 02:36:15.590351 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 537/1068] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:36:15 2022] GET /compute/v2.1/servers/30aa22c4-59b4-4482-b92a-3f386466f7ee => generated 1266 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:36:16.610983 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-29253b7c-69d7-46cf-b256-84b95d5a223d tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:36:16.611608 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-29253b7c-69d7-46cf-b256-84b95d5a223d tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] [instance: 30aa22c4-59b4-4482-b92a-3f386466f7ee] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:36:16.618296 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-29253b7c-69d7-46cf-b256-84b95d5a223d tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:36:16.618822 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-29253b7c-69d7-46cf-b256-84b95d5a223d tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:36:16.619255 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-29253b7c-69d7-46cf-b256-84b95d5a223d tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:36:16.619930 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-29253b7c-69d7-46cf-b256-84b95d5a223d tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:36:16.620381 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-29253b7c-69d7-46cf-b256-84b95d5a223d tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:36:16.620804 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-29253b7c-69d7-46cf-b256-84b95d5a223d tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:36:16.766780 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-29253b7c-69d7-46cf-b256-84b95d5a223d tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3fe5a1a037fa460eb3121e4d307542da', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '79e671941c234f36b542080d256de2c6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:36:16.775010 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-29253b7c-69d7-46cf-b256-84b95d5a223d tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] 10.222.0.52 "GET /compute/v2.1/servers/30aa22c4-59b4-4482-b92a-3f386466f7ee" status: 200 len: 1266 microversion: 2.1 time: 0.168202 Aug 27 02:36:16.775584 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 532/1069] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:36:16 2022] GET /compute/v2.1/servers/30aa22c4-59b4-4482-b92a-3f386466f7ee => generated 1266 bytes in 170 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:36:17.796546 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-b4439345-8371-406f-98df-744c3057e86a tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Calling method '>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:36:17.797364 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-b4439345-8371-406f-98df-744c3057e86a tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] [instance: 30aa22c4-59b4-4482-b92a-3f386466f7ee] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:36:17.803739 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-b4439345-8371-406f-98df-744c3057e86a tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:36:17.804248 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-b4439345-8371-406f-98df-744c3057e86a tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:36:17.804680 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-b4439345-8371-406f-98df-744c3057e86a tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:36:17.805936 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-b4439345-8371-406f-98df-744c3057e86a tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:36:17.805936 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-b4439345-8371-406f-98df-744c3057e86a tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:36:17.806811 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-b4439345-8371-406f-98df-744c3057e86a tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:36:17.948440 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-b4439345-8371-406f-98df-744c3057e86a tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3fe5a1a037fa460eb3121e4d307542da', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '79e671941c234f36b542080d256de2c6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:36:17.957814 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-b4439345-8371-406f-98df-744c3057e86a tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] 10.222.0.52 "GET /compute/v2.1/servers/30aa22c4-59b4-4482-b92a-3f386466f7ee" status: 200 len: 1266 microversion: 2.1 time: 0.165489 Aug 27 02:36:17.958392 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 538/1070] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:36:17 2022] GET /compute/v2.1/servers/30aa22c4-59b4-4482-b92a-3f386466f7ee => generated 1266 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:36:18.987537 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-1b2d7ca5-b3c4-4d5a-9e15-9cbd0ced4a5a tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:36:18.988125 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-1b2d7ca5-b3c4-4d5a-9e15-9cbd0ced4a5a tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] [instance: 30aa22c4-59b4-4482-b92a-3f386466f7ee] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:36:18.994933 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-1b2d7ca5-b3c4-4d5a-9e15-9cbd0ced4a5a tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:36:18.995437 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-1b2d7ca5-b3c4-4d5a-9e15-9cbd0ced4a5a tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:36:18.995878 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-1b2d7ca5-b3c4-4d5a-9e15-9cbd0ced4a5a tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:36:18.996787 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-1b2d7ca5-b3c4-4d5a-9e15-9cbd0ced4a5a tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:36:18.997098 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-1b2d7ca5-b3c4-4d5a-9e15-9cbd0ced4a5a tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:36:18.997586 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-1b2d7ca5-b3c4-4d5a-9e15-9cbd0ced4a5a tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:36:19.070307 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [req-c512a97c-1822-4033-89dc-f91d0bf0ebe1 req-287bcc86-405c-4775-8357-d08c615e5936 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "30aa22c4-59b4-4482-b92a-3f386466f7ee", "name": "network-vif-plugged", "status": "completed", "tag": "d3b3018c-6150-428a-a235-9ccd41ba80e7"}]} {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 27 02:36:19.082566 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [req-c512a97c-1822-4033-89dc-f91d0bf0ebe1 req-287bcc86-405c-4775-8357-d08c615e5936 service nova] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:36:19.083280 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [req-c512a97c-1822-4033-89dc-f91d0bf0ebe1 req-287bcc86-405c-4775-8357-d08c615e5936 service nova] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:36:19.083875 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [req-c512a97c-1822-4033-89dc-f91d0bf0ebe1 req-287bcc86-405c-4775-8357-d08c615e5936 service nova] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:36:19.098112 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.compute.server_external_events [req-c512a97c-1822-4033-89dc-f91d0bf0ebe1 req-287bcc86-405c-4775-8357-d08c615e5936 service nova] Creating event network-vif-plugged:d3b3018c-6150-428a-a235-9ccd41ba80e7 for instance 30aa22c4-59b4-4482-b92a-3f386466f7ee on hv-cinder-84630 Aug 27 02:36:19.103864 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [req-c512a97c-1822-4033-89dc-f91d0bf0ebe1 req-287bcc86-405c-4775-8357-d08c615e5936 service nova] 10.222.0.52 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.037648 Aug 27 02:36:19.104852 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 539/1071] 10.222.0.52 () {64 vars in 1317 bytes} [Sat Aug 27 02:36:19 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 40 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 27 02:36:19.143871 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-1b2d7ca5-b3c4-4d5a-9e15-9cbd0ced4a5a tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3fe5a1a037fa460eb3121e4d307542da', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '79e671941c234f36b542080d256de2c6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:36:19.152306 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-1b2d7ca5-b3c4-4d5a-9e15-9cbd0ced4a5a tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] 10.222.0.52 "GET /compute/v2.1/servers/30aa22c4-59b4-4482-b92a-3f386466f7ee" status: 200 len: 1266 microversion: 2.1 time: 0.168688 Aug 27 02:36:19.152889 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 533/1072] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:36:18 2022] GET /compute/v2.1/servers/30aa22c4-59b4-4482-b92a-3f386466f7ee => generated 1266 bytes in 170 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:36:20.174925 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-e42a04f2-e461-488b-b266-8aafdabe7704 tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Calling method '>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:36:20.175590 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-e42a04f2-e461-488b-b266-8aafdabe7704 tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] [instance: 30aa22c4-59b4-4482-b92a-3f386466f7ee] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:36:20.183432 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-e42a04f2-e461-488b-b266-8aafdabe7704 tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:36:20.183915 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-e42a04f2-e461-488b-b266-8aafdabe7704 tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:36:20.184349 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-e42a04f2-e461-488b-b266-8aafdabe7704 tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:36:20.185041 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-e42a04f2-e461-488b-b266-8aafdabe7704 tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:36:20.185532 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-e42a04f2-e461-488b-b266-8aafdabe7704 tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:36:20.185941 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-e42a04f2-e461-488b-b266-8aafdabe7704 tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:36:20.335740 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-e42a04f2-e461-488b-b266-8aafdabe7704 tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3fe5a1a037fa460eb3121e4d307542da', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '79e671941c234f36b542080d256de2c6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:36:20.349320 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-e42a04f2-e461-488b-b266-8aafdabe7704 tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] 10.222.0.52 "GET /compute/v2.1/servers/30aa22c4-59b4-4482-b92a-3f386466f7ee" status: 200 len: 1266 microversion: 2.1 time: 0.178647 Aug 27 02:36:20.350188 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 540/1073] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:36:20 2022] GET /compute/v2.1/servers/30aa22c4-59b4-4482-b92a-3f386466f7ee => generated 1266 bytes in 181 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:36:21.374303 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-cf0a23f5-e672-4266-bc63-d4b7e8489385 tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:36:21.374902 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-cf0a23f5-e672-4266-bc63-d4b7e8489385 tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] [instance: 30aa22c4-59b4-4482-b92a-3f386466f7ee] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:36:21.382307 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-cf0a23f5-e672-4266-bc63-d4b7e8489385 tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:36:21.382803 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-cf0a23f5-e672-4266-bc63-d4b7e8489385 tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:36:21.383233 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-cf0a23f5-e672-4266-bc63-d4b7e8489385 tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:36:21.383940 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-cf0a23f5-e672-4266-bc63-d4b7e8489385 tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:36:21.384395 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-cf0a23f5-e672-4266-bc63-d4b7e8489385 tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:36:21.385015 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-cf0a23f5-e672-4266-bc63-d4b7e8489385 tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:36:21.535726 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-cf0a23f5-e672-4266-bc63-d4b7e8489385 tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3fe5a1a037fa460eb3121e4d307542da', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '79e671941c234f36b542080d256de2c6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:36:21.543998 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-cf0a23f5-e672-4266-bc63-d4b7e8489385 tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] 10.222.0.52 "GET /compute/v2.1/servers/30aa22c4-59b4-4482-b92a-3f386466f7ee" status: 200 len: 1454 microversion: 2.1 time: 0.173573 Aug 27 02:36:21.544573 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 534/1074] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:36:21 2022] GET /compute/v2.1/servers/30aa22c4-59b4-4482-b92a-3f386466f7ee => generated 1454 bytes in 175 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:36:21.566442 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-eaf1c9ca-64cd-468a-924c-72ca148b30fa tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Calling method '>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:36:21.567161 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-eaf1c9ca-64cd-468a-924c-72ca148b30fa tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] [instance: 30aa22c4-59b4-4482-b92a-3f386466f7ee] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:36:21.575203 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-eaf1c9ca-64cd-468a-924c-72ca148b30fa tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:36:21.575722 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-eaf1c9ca-64cd-468a-924c-72ca148b30fa tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:36:21.576190 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-eaf1c9ca-64cd-468a-924c-72ca148b30fa tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:36:21.576926 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-eaf1c9ca-64cd-468a-924c-72ca148b30fa tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:36:21.577417 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-eaf1c9ca-64cd-468a-924c-72ca148b30fa tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:36:21.577833 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-eaf1c9ca-64cd-468a-924c-72ca148b30fa tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:36:21.740346 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-eaf1c9ca-64cd-468a-924c-72ca148b30fa tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3fe5a1a037fa460eb3121e4d307542da', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '79e671941c234f36b542080d256de2c6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:36:21.748917 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-eaf1c9ca-64cd-468a-924c-72ca148b30fa tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] 10.222.0.52 "GET /compute/v2.1/servers/30aa22c4-59b4-4482-b92a-3f386466f7ee" status: 200 len: 1454 microversion: 2.1 time: 0.187280 Aug 27 02:36:21.749510 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 541/1075] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:36:21 2022] GET /compute/v2.1/servers/30aa22c4-59b4-4482-b92a-3f386466f7ee => generated 1454 bytes in 189 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:36:23.818527 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [req-2672b99e-ef50-4023-a3f5-6e8fc4a61d9b req-c1289138-186e-43fa-b8c6-52d6026fac46 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "30aa22c4-59b4-4482-b92a-3f386466f7ee", "tag": "d3b3018c-6150-428a-a235-9ccd41ba80e7"}]} {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 27 02:36:23.829299 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [req-2672b99e-ef50-4023-a3f5-6e8fc4a61d9b req-c1289138-186e-43fa-b8c6-52d6026fac46 service nova] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:36:23.829785 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [req-2672b99e-ef50-4023-a3f5-6e8fc4a61d9b req-c1289138-186e-43fa-b8c6-52d6026fac46 service nova] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:36:23.830209 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [req-2672b99e-ef50-4023-a3f5-6e8fc4a61d9b req-c1289138-186e-43fa-b8c6-52d6026fac46 service nova] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:36:23.842880 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.compute.server_external_events [req-2672b99e-ef50-4023-a3f5-6e8fc4a61d9b req-c1289138-186e-43fa-b8c6-52d6026fac46 service nova] Creating event network-changed:d3b3018c-6150-428a-a235-9ccd41ba80e7 for instance 30aa22c4-59b4-4482-b92a-3f386466f7ee on hv-cinder-84630 Aug 27 02:36:23.854132 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [req-2672b99e-ef50-4023-a3f5-6e8fc4a61d9b req-c1289138-186e-43fa-b8c6-52d6026fac46 service nova] 10.222.0.52 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.038854 Aug 27 02:36:23.854923 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 535/1076] 10.222.0.52 () {64 vars in 1317 bytes} [Sat Aug 27 02:36:23 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 42 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 27 02:37:07.683676 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-14b09ced-4750-453d-a569-c7e709ea8200 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Calling method '>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:37:07.684360 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-14b09ced-4750-453d-a569-c7e709ea8200 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] [instance: f5bad70c-350c-4230-a862-626dd6805e4c] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:37:07.686825 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG dbcounter [-] [120864] Writing DB stats nova_api:SELECT=22 {{(pid=120864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 27 02:37:07.687574 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG dbcounter [-] [120864] Writing DB stats nova_cell1:SELECT=50 {{(pid=120864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 27 02:37:07.696955 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-14b09ced-4750-453d-a569-c7e709ea8200 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:37:07.697479 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-14b09ced-4750-453d-a569-c7e709ea8200 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:37:07.698174 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-14b09ced-4750-453d-a569-c7e709ea8200 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:37:07.699335 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-14b09ced-4750-453d-a569-c7e709ea8200 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:37:07.700093 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-14b09ced-4750-453d-a569-c7e709ea8200 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:37:07.700799 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-14b09ced-4750-453d-a569-c7e709ea8200 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:37:07.732412 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-14b09ced-4750-453d-a569-c7e709ea8200 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] 10.222.0.52 "GET /compute/v2.1/servers/f5bad70c-350c-4230-a862-626dd6805e4c/ips" status: 200 len: 144 microversion: 2.1 time: 0.052780 Aug 27 02:37:07.733003 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 542/1077] 10.222.0.52 () {58 vars in 1248 bytes} [Sat Aug 27 02:37:07 2022] GET /compute/v2.1/servers/f5bad70c-350c-4230-a862-626dd6805e4c/ips => generated 144 bytes in 54 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 27 02:37:10.805073 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG dbcounter [-] [120863] Writing DB stats nova_api:SELECT=33,nova_api:INSERT=3 {{(pid=120863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 27 02:37:10.805897 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG dbcounter [-] [120863] Writing DB stats nova_cell1:SELECT=48 {{(pid=120863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 27 02:37:10.811000 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-6b65fb44-eeec-400f-a6d2-a48760a9ff39 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "f63d05dc-0d65-4bb4-b351-74e7517eae22"}} {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 27 02:37:10.813486 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-6b65fb44-eeec-400f-a6d2-a48760a9ff39 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] [instance: f5bad70c-350c-4230-a862-626dd6805e4c] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:37:10.821278 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-6b65fb44-eeec-400f-a6d2-a48760a9ff39 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:37:10.821755 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-6b65fb44-eeec-400f-a6d2-a48760a9ff39 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:37:10.822278 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-6b65fb44-eeec-400f-a6d2-a48760a9ff39 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:37:10.823161 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-6b65fb44-eeec-400f-a6d2-a48760a9ff39 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:37:10.823608 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-6b65fb44-eeec-400f-a6d2-a48760a9ff39 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:37:10.824026 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-6b65fb44-eeec-400f-a6d2-a48760a9ff39 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:37:11.111914 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-6b65fb44-eeec-400f-a6d2-a48760a9ff39 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] 10.222.0.52 "POST /compute/v2.1/servers/f5bad70c-350c-4230-a862-626dd6805e4c/os-volume_attachments" status: 200 len: 194 microversion: 2.1 time: 0.308587 Aug 27 02:37:11.112597 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 536/1078] 10.222.0.52 () {60 vars in 1323 bytes} [Sat Aug 27 02:37:10 2022] POST /compute/v2.1/servers/f5bad70c-350c-4230-a862-626dd6805e4c/os-volume_attachments => generated 194 bytes in 310 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 27 02:37:18.705981 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-4da0f98a-7a3a-4742-9b60-6003e5499e8e tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Action: 'action', calling method: >, body: {"os-stop": {}} {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 27 02:37:18.708846 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG dbcounter [-] [120864] Writing DB stats nova_api:SELECT=2 {{(pid=120864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 27 02:37:18.709553 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG dbcounter [-] [120864] Writing DB stats nova_cell1:SELECT=4 {{(pid=120864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 27 02:37:18.716722 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-4da0f98a-7a3a-4742-9b60-6003e5499e8e tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:37:18.717273 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-4da0f98a-7a3a-4742-9b60-6003e5499e8e tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:37:18.717748 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-4da0f98a-7a3a-4742-9b60-6003e5499e8e tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:37:18.779679 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-4da0f98a-7a3a-4742-9b60-6003e5499e8e tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] [instance: f5bad70c-350c-4230-a862-626dd6805e4c] Going to try to stop instance {{(pid=120864) force_stop /opt/stack/nova/nova/compute/api.py:2762}} Aug 27 02:37:18.829400 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-4da0f98a-7a3a-4742-9b60-6003e5499e8e tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] 10.222.0.52 "POST /compute/v2.1/servers/f5bad70c-350c-4230-a862-626dd6805e4c/action" status: 202 len: 0 microversion: 2.1 time: 0.127324 Aug 27 02:37:18.829929 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 543/1079] 10.222.0.52 () {60 vars in 1278 bytes} [Sat Aug 27 02:37:18 2022] POST /compute/v2.1/servers/f5bad70c-350c-4230-a862-626dd6805e4c/action => generated 0 bytes in 129 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Aug 27 02:37:18.842662 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-47a2e175-5b1f-4251-8d4a-1026ce010d89 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:37:18.843808 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-47a2e175-5b1f-4251-8d4a-1026ce010d89 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] [instance: f5bad70c-350c-4230-a862-626dd6805e4c] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:37:18.850099 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-47a2e175-5b1f-4251-8d4a-1026ce010d89 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:37:18.850583 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-47a2e175-5b1f-4251-8d4a-1026ce010d89 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:37:18.851197 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-47a2e175-5b1f-4251-8d4a-1026ce010d89 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:37:18.851911 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-47a2e175-5b1f-4251-8d4a-1026ce010d89 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:37:18.852359 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-47a2e175-5b1f-4251-8d4a-1026ce010d89 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:37:18.852806 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-47a2e175-5b1f-4251-8d4a-1026ce010d89 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:37:18.995725 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-47a2e175-5b1f-4251-8d4a-1026ce010d89 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee6084fffc544dfe8f1a9a30ae06be69', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a8055f4326f640fc8927f5cfdef39dd3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:37:19.007696 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-47a2e175-5b1f-4251-8d4a-1026ce010d89 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] 10.222.0.52 "GET /compute/v2.1/servers/f5bad70c-350c-4230-a862-626dd6805e4c" status: 200 len: 1722 microversion: 2.1 time: 0.171240 Aug 27 02:37:19.008671 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 537/1080] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:37:18 2022] GET /compute/v2.1/servers/f5bad70c-350c-4230-a862-626dd6805e4c => generated 1722 bytes in 174 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:37:20.032061 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-d1be787b-c949-446b-97a1-3d6ab96ac859 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Calling method '>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:37:20.032638 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-d1be787b-c949-446b-97a1-3d6ab96ac859 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] [instance: f5bad70c-350c-4230-a862-626dd6805e4c] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:37:20.039365 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-d1be787b-c949-446b-97a1-3d6ab96ac859 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:37:20.039853 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-d1be787b-c949-446b-97a1-3d6ab96ac859 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:37:20.040290 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-d1be787b-c949-446b-97a1-3d6ab96ac859 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:37:20.040983 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-d1be787b-c949-446b-97a1-3d6ab96ac859 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:37:20.041470 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-d1be787b-c949-446b-97a1-3d6ab96ac859 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:37:20.041902 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-d1be787b-c949-446b-97a1-3d6ab96ac859 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:37:20.185226 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-d1be787b-c949-446b-97a1-3d6ab96ac859 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee6084fffc544dfe8f1a9a30ae06be69', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a8055f4326f640fc8927f5cfdef39dd3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:37:20.200209 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-d1be787b-c949-446b-97a1-3d6ab96ac859 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] 10.222.0.52 "GET /compute/v2.1/servers/f5bad70c-350c-4230-a862-626dd6805e4c" status: 200 len: 1722 microversion: 2.1 time: 0.172936 Aug 27 02:37:20.200850 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 544/1081] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:37:20 2022] GET /compute/v2.1/servers/f5bad70c-350c-4230-a862-626dd6805e4c => generated 1722 bytes in 174 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:37:21.226290 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-0338e834-1744-4569-a042-f4c9a64c84b1 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:37:21.226939 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-0338e834-1744-4569-a042-f4c9a64c84b1 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] [instance: f5bad70c-350c-4230-a862-626dd6805e4c] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:37:21.235733 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-0338e834-1744-4569-a042-f4c9a64c84b1 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:37:21.236213 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-0338e834-1744-4569-a042-f4c9a64c84b1 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:37:21.236644 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-0338e834-1744-4569-a042-f4c9a64c84b1 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:37:21.237397 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-0338e834-1744-4569-a042-f4c9a64c84b1 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:37:21.237860 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-0338e834-1744-4569-a042-f4c9a64c84b1 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:37:21.238275 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-0338e834-1744-4569-a042-f4c9a64c84b1 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:37:21.382621 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-0338e834-1744-4569-a042-f4c9a64c84b1 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee6084fffc544dfe8f1a9a30ae06be69', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a8055f4326f640fc8927f5cfdef39dd3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:37:21.392425 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-0338e834-1744-4569-a042-f4c9a64c84b1 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] 10.222.0.52 "GET /compute/v2.1/servers/f5bad70c-350c-4230-a862-626dd6805e4c" status: 200 len: 1722 microversion: 2.1 time: 0.172906 Aug 27 02:37:21.392964 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 538/1082] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:37:21 2022] GET /compute/v2.1/servers/f5bad70c-350c-4230-a862-626dd6805e4c => generated 1722 bytes in 174 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:37:22.417705 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-3cf3bea8-8aa9-4118-b04a-32e3f416d173 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Calling method '>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:37:22.418661 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-3cf3bea8-8aa9-4118-b04a-32e3f416d173 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] [instance: f5bad70c-350c-4230-a862-626dd6805e4c] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:37:22.429100 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-3cf3bea8-8aa9-4118-b04a-32e3f416d173 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:37:22.429891 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-3cf3bea8-8aa9-4118-b04a-32e3f416d173 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:37:22.430636 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-3cf3bea8-8aa9-4118-b04a-32e3f416d173 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:37:22.431672 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-3cf3bea8-8aa9-4118-b04a-32e3f416d173 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:37:22.432272 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-3cf3bea8-8aa9-4118-b04a-32e3f416d173 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:37:22.432717 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-3cf3bea8-8aa9-4118-b04a-32e3f416d173 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:37:22.571688 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-3cf3bea8-8aa9-4118-b04a-32e3f416d173 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee6084fffc544dfe8f1a9a30ae06be69', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a8055f4326f640fc8927f5cfdef39dd3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:37:22.584786 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-3cf3bea8-8aa9-4118-b04a-32e3f416d173 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] 10.222.0.52 "GET /compute/v2.1/servers/f5bad70c-350c-4230-a862-626dd6805e4c" status: 200 len: 1722 microversion: 2.1 time: 0.173290 Aug 27 02:37:22.585597 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 545/1083] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:37:22 2022] GET /compute/v2.1/servers/f5bad70c-350c-4230-a862-626dd6805e4c => generated 1722 bytes in 175 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:37:23.611816 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-a4b7211a-b1c5-4456-82e2-bebaa5ddde12 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:37:23.612573 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-a4b7211a-b1c5-4456-82e2-bebaa5ddde12 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] [instance: f5bad70c-350c-4230-a862-626dd6805e4c] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:37:23.621477 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-a4b7211a-b1c5-4456-82e2-bebaa5ddde12 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:37:23.621977 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-a4b7211a-b1c5-4456-82e2-bebaa5ddde12 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:37:23.622633 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-a4b7211a-b1c5-4456-82e2-bebaa5ddde12 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:37:23.623375 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-a4b7211a-b1c5-4456-82e2-bebaa5ddde12 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:37:23.624293 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-a4b7211a-b1c5-4456-82e2-bebaa5ddde12 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:37:23.624440 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-a4b7211a-b1c5-4456-82e2-bebaa5ddde12 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:37:23.774722 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-a4b7211a-b1c5-4456-82e2-bebaa5ddde12 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee6084fffc544dfe8f1a9a30ae06be69', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a8055f4326f640fc8927f5cfdef39dd3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:37:23.784110 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-a4b7211a-b1c5-4456-82e2-bebaa5ddde12 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] 10.222.0.52 "GET /compute/v2.1/servers/f5bad70c-350c-4230-a862-626dd6805e4c" status: 200 len: 1722 microversion: 2.1 time: 0.177843 Aug 27 02:37:23.784770 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 539/1084] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:37:23 2022] GET /compute/v2.1/servers/f5bad70c-350c-4230-a862-626dd6805e4c => generated 1722 bytes in 179 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:37:24.812218 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-9fd58b93-be59-41be-8282-677e94e66530 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Calling method '>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:37:24.812783 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-9fd58b93-be59-41be-8282-677e94e66530 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] [instance: f5bad70c-350c-4230-a862-626dd6805e4c] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:37:24.820219 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-9fd58b93-be59-41be-8282-677e94e66530 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:37:24.820743 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-9fd58b93-be59-41be-8282-677e94e66530 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:37:24.821166 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-9fd58b93-be59-41be-8282-677e94e66530 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:37:24.821903 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-9fd58b93-be59-41be-8282-677e94e66530 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:37:24.822347 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-9fd58b93-be59-41be-8282-677e94e66530 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:37:24.822765 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-9fd58b93-be59-41be-8282-677e94e66530 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:37:24.978715 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-9fd58b93-be59-41be-8282-677e94e66530 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee6084fffc544dfe8f1a9a30ae06be69', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a8055f4326f640fc8927f5cfdef39dd3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:37:24.987926 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-9fd58b93-be59-41be-8282-677e94e66530 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] 10.222.0.52 "GET /compute/v2.1/servers/f5bad70c-350c-4230-a862-626dd6805e4c" status: 200 len: 1722 microversion: 2.1 time: 0.181603 Aug 27 02:37:24.988505 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 546/1085] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:37:24 2022] GET /compute/v2.1/servers/f5bad70c-350c-4230-a862-626dd6805e4c => generated 1722 bytes in 183 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:37:26.010771 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-be6c8ef3-718e-4dc8-af2b-d6292812b23b tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:37:26.011373 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-be6c8ef3-718e-4dc8-af2b-d6292812b23b tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] [instance: f5bad70c-350c-4230-a862-626dd6805e4c] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:37:26.018642 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-be6c8ef3-718e-4dc8-af2b-d6292812b23b tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:37:26.019126 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-be6c8ef3-718e-4dc8-af2b-d6292812b23b tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:37:26.019564 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-be6c8ef3-718e-4dc8-af2b-d6292812b23b tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:37:26.020260 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-be6c8ef3-718e-4dc8-af2b-d6292812b23b tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:37:26.020716 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-be6c8ef3-718e-4dc8-af2b-d6292812b23b tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:37:26.021147 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-be6c8ef3-718e-4dc8-af2b-d6292812b23b tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:37:26.175224 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-be6c8ef3-718e-4dc8-af2b-d6292812b23b tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee6084fffc544dfe8f1a9a30ae06be69', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a8055f4326f640fc8927f5cfdef39dd3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:37:26.185286 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-be6c8ef3-718e-4dc8-af2b-d6292812b23b tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] 10.222.0.52 "GET /compute/v2.1/servers/f5bad70c-350c-4230-a862-626dd6805e4c" status: 200 len: 1722 microversion: 2.1 time: 0.178233 Aug 27 02:37:26.185902 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 540/1086] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:37:26 2022] GET /compute/v2.1/servers/f5bad70c-350c-4230-a862-626dd6805e4c => generated 1722 bytes in 180 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 27 02:37:27.214869 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-05baa78f-8f0f-420b-b95f-00ef14ee5b00 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Calling method '>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:37:27.215709 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-05baa78f-8f0f-420b-b95f-00ef14ee5b00 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] [instance: f5bad70c-350c-4230-a862-626dd6805e4c] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:37:27.224926 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-05baa78f-8f0f-420b-b95f-00ef14ee5b00 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:37:27.225607 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-05baa78f-8f0f-420b-b95f-00ef14ee5b00 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:37:27.226172 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-05baa78f-8f0f-420b-b95f-00ef14ee5b00 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:37:27.227076 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-05baa78f-8f0f-420b-b95f-00ef14ee5b00 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:37:27.227665 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-05baa78f-8f0f-420b-b95f-00ef14ee5b00 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:37:27.228214 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-05baa78f-8f0f-420b-b95f-00ef14ee5b00 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:37:27.396084 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-05baa78f-8f0f-420b-b95f-00ef14ee5b00 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee6084fffc544dfe8f1a9a30ae06be69', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a8055f4326f640fc8927f5cfdef39dd3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:37:27.405218 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-05baa78f-8f0f-420b-b95f-00ef14ee5b00 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] 10.222.0.52 "GET /compute/v2.1/servers/f5bad70c-350c-4230-a862-626dd6805e4c" status: 200 len: 1722 microversion: 2.1 time: 0.195599 Aug 27 02:37:27.405875 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 547/1087] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:37:27 2022] GET /compute/v2.1/servers/f5bad70c-350c-4230-a862-626dd6805e4c => generated 1722 bytes in 197 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:37:28.430727 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-eb7bd4b6-c1a5-4bce-99bd-60c51ffe7a00 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:37:28.431702 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-eb7bd4b6-c1a5-4bce-99bd-60c51ffe7a00 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] [instance: f5bad70c-350c-4230-a862-626dd6805e4c] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:37:28.440380 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-eb7bd4b6-c1a5-4bce-99bd-60c51ffe7a00 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:37:28.440903 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-eb7bd4b6-c1a5-4bce-99bd-60c51ffe7a00 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:37:28.441377 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-eb7bd4b6-c1a5-4bce-99bd-60c51ffe7a00 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:37:28.442070 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-eb7bd4b6-c1a5-4bce-99bd-60c51ffe7a00 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:37:28.442518 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-eb7bd4b6-c1a5-4bce-99bd-60c51ffe7a00 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:37:28.442936 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-eb7bd4b6-c1a5-4bce-99bd-60c51ffe7a00 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:37:28.597564 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-eb7bd4b6-c1a5-4bce-99bd-60c51ffe7a00 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee6084fffc544dfe8f1a9a30ae06be69', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a8055f4326f640fc8927f5cfdef39dd3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:37:28.609721 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-eb7bd4b6-c1a5-4bce-99bd-60c51ffe7a00 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] 10.222.0.52 "GET /compute/v2.1/servers/f5bad70c-350c-4230-a862-626dd6805e4c" status: 200 len: 1722 microversion: 2.1 time: 0.185035 Aug 27 02:37:28.610308 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 541/1088] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:37:28 2022] GET /compute/v2.1/servers/f5bad70c-350c-4230-a862-626dd6805e4c => generated 1722 bytes in 187 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:37:29.634159 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-53942191-5596-46ae-a930-ee48c4dd67f1 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Calling method '>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:37:29.634855 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-53942191-5596-46ae-a930-ee48c4dd67f1 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] [instance: f5bad70c-350c-4230-a862-626dd6805e4c] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:37:29.643259 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-53942191-5596-46ae-a930-ee48c4dd67f1 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:37:29.643766 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-53942191-5596-46ae-a930-ee48c4dd67f1 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:37:29.644200 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-53942191-5596-46ae-a930-ee48c4dd67f1 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:37:29.644887 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-53942191-5596-46ae-a930-ee48c4dd67f1 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:37:29.645452 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-53942191-5596-46ae-a930-ee48c4dd67f1 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:37:29.645869 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-53942191-5596-46ae-a930-ee48c4dd67f1 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:37:29.809993 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-53942191-5596-46ae-a930-ee48c4dd67f1 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee6084fffc544dfe8f1a9a30ae06be69', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a8055f4326f640fc8927f5cfdef39dd3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:37:29.819690 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-53942191-5596-46ae-a930-ee48c4dd67f1 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] 10.222.0.52 "GET /compute/v2.1/servers/f5bad70c-350c-4230-a862-626dd6805e4c" status: 200 len: 1699 microversion: 2.1 time: 0.191402 Aug 27 02:37:29.820242 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 548/1089] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:37:29 2022] GET /compute/v2.1/servers/f5bad70c-350c-4230-a862-626dd6805e4c => generated 1699 bytes in 193 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:37:29.841943 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-a6b9db0b-ae0b-4f0d-a5bb-92ae88840321 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Action: 'action', calling method: >, body: {"os-start": {}} {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 27 02:37:29.849346 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-a6b9db0b-ae0b-4f0d-a5bb-92ae88840321 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:37:29.849842 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-a6b9db0b-ae0b-4f0d-a5bb-92ae88840321 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:37:29.850281 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-a6b9db0b-ae0b-4f0d-a5bb-92ae88840321 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:37:29.912050 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-a6b9db0b-ae0b-4f0d-a5bb-92ae88840321 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] [instance: f5bad70c-350c-4230-a862-626dd6805e4c] Going to try to start instance {{(pid=120863) start /opt/stack/nova/nova/compute/api.py:2785}} Aug 27 02:37:29.958763 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-a6b9db0b-ae0b-4f0d-a5bb-92ae88840321 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] 10.222.0.52 "POST /compute/v2.1/servers/f5bad70c-350c-4230-a862-626dd6805e4c/action" status: 202 len: 0 microversion: 2.1 time: 0.121384 Aug 27 02:37:29.959352 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 542/1090] 10.222.0.52 () {60 vars in 1278 bytes} [Sat Aug 27 02:37:29 2022] POST /compute/v2.1/servers/f5bad70c-350c-4230-a862-626dd6805e4c/action => generated 0 bytes in 123 msecs (HTTP/1.1 202) 9 headers in 361 bytes (2 switches on core 0) Aug 27 02:37:29.969398 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-5db51021-f311-4fd8-bc0e-94fa9af78edd tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Calling method '>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:37:29.970145 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-5db51021-f311-4fd8-bc0e-94fa9af78edd tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] [instance: f5bad70c-350c-4230-a862-626dd6805e4c] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:37:29.976385 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-5db51021-f311-4fd8-bc0e-94fa9af78edd tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:37:29.976904 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-5db51021-f311-4fd8-bc0e-94fa9af78edd tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:37:29.977380 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-5db51021-f311-4fd8-bc0e-94fa9af78edd tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:37:29.978079 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-5db51021-f311-4fd8-bc0e-94fa9af78edd tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:37:29.978533 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-5db51021-f311-4fd8-bc0e-94fa9af78edd tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:37:29.978950 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-5db51021-f311-4fd8-bc0e-94fa9af78edd tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:37:30.125725 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-5db51021-f311-4fd8-bc0e-94fa9af78edd tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee6084fffc544dfe8f1a9a30ae06be69', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a8055f4326f640fc8927f5cfdef39dd3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:37:30.134517 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-5db51021-f311-4fd8-bc0e-94fa9af78edd tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] 10.222.0.52 "GET /compute/v2.1/servers/f5bad70c-350c-4230-a862-626dd6805e4c" status: 200 len: 1708 microversion: 2.1 time: 0.169727 Aug 27 02:37:30.135095 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 549/1091] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:37:29 2022] GET /compute/v2.1/servers/f5bad70c-350c-4230-a862-626dd6805e4c => generated 1708 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:37:31.161681 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-5f7bf7e8-1a4e-4132-90f0-a0d645047e89 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:37:31.162527 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-5f7bf7e8-1a4e-4132-90f0-a0d645047e89 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] [instance: f5bad70c-350c-4230-a862-626dd6805e4c] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:37:31.171979 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-5f7bf7e8-1a4e-4132-90f0-a0d645047e89 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:37:31.172453 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-5f7bf7e8-1a4e-4132-90f0-a0d645047e89 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:37:31.172882 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-5f7bf7e8-1a4e-4132-90f0-a0d645047e89 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:37:31.173603 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-5f7bf7e8-1a4e-4132-90f0-a0d645047e89 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:37:31.174051 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-5f7bf7e8-1a4e-4132-90f0-a0d645047e89 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:37:31.174468 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-5f7bf7e8-1a4e-4132-90f0-a0d645047e89 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:37:31.317060 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-5f7bf7e8-1a4e-4132-90f0-a0d645047e89 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee6084fffc544dfe8f1a9a30ae06be69', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a8055f4326f640fc8927f5cfdef39dd3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:37:31.325418 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-5f7bf7e8-1a4e-4132-90f0-a0d645047e89 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] 10.222.0.52 "GET /compute/v2.1/servers/f5bad70c-350c-4230-a862-626dd6805e4c" status: 200 len: 1708 microversion: 2.1 time: 0.171695 Aug 27 02:37:31.325993 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 543/1092] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:37:31 2022] GET /compute/v2.1/servers/f5bad70c-350c-4230-a862-626dd6805e4c => generated 1708 bytes in 174 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:37:32.347717 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-de9bfc2d-6d02-4831-9986-60be75f01314 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Calling method '>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:37:32.348367 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-de9bfc2d-6d02-4831-9986-60be75f01314 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] [instance: f5bad70c-350c-4230-a862-626dd6805e4c] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:37:32.355656 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-de9bfc2d-6d02-4831-9986-60be75f01314 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:37:32.356121 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-de9bfc2d-6d02-4831-9986-60be75f01314 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:37:32.356550 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-de9bfc2d-6d02-4831-9986-60be75f01314 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:37:32.357326 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-de9bfc2d-6d02-4831-9986-60be75f01314 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:37:32.357839 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-de9bfc2d-6d02-4831-9986-60be75f01314 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:37:32.358260 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-de9bfc2d-6d02-4831-9986-60be75f01314 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:37:32.508499 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-de9bfc2d-6d02-4831-9986-60be75f01314 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee6084fffc544dfe8f1a9a30ae06be69', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a8055f4326f640fc8927f5cfdef39dd3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:37:32.517856 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-de9bfc2d-6d02-4831-9986-60be75f01314 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] 10.222.0.52 "GET /compute/v2.1/servers/f5bad70c-350c-4230-a862-626dd6805e4c" status: 200 len: 1712 microversion: 2.1 time: 0.174024 Aug 27 02:37:32.518430 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 550/1093] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:37:32 2022] GET /compute/v2.1/servers/f5bad70c-350c-4230-a862-626dd6805e4c => generated 1712 bytes in 175 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:38:35.727152 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG dbcounter [-] [120863] Writing DB stats nova_api:SELECT=16 {{(pid=120863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 27 02:38:35.728087 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG dbcounter [-] [120863] Writing DB stats nova_cell1:SELECT=57,nova_cell1:UPDATE=3,nova_cell1:INSERT=2 {{(pid=120863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 27 02:38:35.729191 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Aug 27 02:38:35.729425 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: Sat Aug 27 02:38:35 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/f5bad70c-350c-4230-a862-626dd6805e4c/os-volume_attachments/f63d05dc-0d65-4bb4-b351-74e7517eae22 (ip 10.222.0.52) !!! Aug 27 02:38:35.730430 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Aug 27 02:38:35.730566 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: Sat Aug 27 02:38:35 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/f5bad70c-350c-4230-a862-626dd6805e4c/os-volume_attachments/f63d05dc-0d65-4bb4-b351-74e7517eae22 (ip 10.222.0.52) !!! Aug 27 02:38:35.736143 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-93ad8de6-b7e4-4106-9e3f-ea88a56afb61 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:38:35.736618 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-93ad8de6-b7e4-4106-9e3f-ea88a56afb61 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] [instance: f5bad70c-350c-4230-a862-626dd6805e4c] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:38:35.747160 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-93ad8de6-b7e4-4106-9e3f-ea88a56afb61 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:38:35.747653 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-93ad8de6-b7e4-4106-9e3f-ea88a56afb61 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:38:35.748107 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-93ad8de6-b7e4-4106-9e3f-ea88a56afb61 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:38:35.748825 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-93ad8de6-b7e4-4106-9e3f-ea88a56afb61 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:38:35.749334 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-93ad8de6-b7e4-4106-9e3f-ea88a56afb61 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:38:35.750083 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-93ad8de6-b7e4-4106-9e3f-ea88a56afb61 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:38:35.948729 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.objects.instance [None req-93ad8de6-b7e4-4106-9e3f-ea88a56afb61 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Lazy-loading 'services' on Instance uuid f5bad70c-350c-4230-a862-626dd6805e4c {{(pid=120863) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Aug 27 02:38:36.028650 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-93ad8de6-b7e4-4106-9e3f-ea88a56afb61 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] 10.222.0.52 "DELETE /compute/v2.1/servers/f5bad70c-350c-4230-a862-626dd6805e4c/os-volume_attachments/f63d05dc-0d65-4bb4-b351-74e7517eae22" status: 202 len: 0 microversion: 2.1 time: 0.302971 Aug 27 02:38:36.030019 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 544/1094] 10.222.0.52 () {58 vars in 1416 bytes} [Sat Aug 27 02:38:35 2022] DELETE /compute/v2.1/servers/f5bad70c-350c-4230-a862-626dd6805e4c/os-volume_attachments/f63d05dc-0d65-4bb4-b351-74e7517eae22 => generated 0 bytes in 305 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Aug 27 02:38:37.156714 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG dbcounter [-] [120864] Writing DB stats nova_api:SELECT=16 {{(pid=120864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 27 02:38:37.157590 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG dbcounter [-] [120864] Writing DB stats nova_cell1:SELECT=52,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=120864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 27 02:38:37.158722 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Aug 27 02:38:37.158898 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: Sat Aug 27 02:38:37 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/f5bad70c-350c-4230-a862-626dd6805e4c/action (ip 10.222.0.52) !!! Aug 27 02:38:37.162620 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Aug 27 02:38:37.162792 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: Sat Aug 27 02:38:37 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/f5bad70c-350c-4230-a862-626dd6805e4c/action (ip 10.222.0.52) !!! Aug 27 02:38:37.174066 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-4df48862-0ba2-4560-8353-72c1c995b9ff tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Action: 'action', calling method: >, body: {"os-stop": {}} {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 27 02:38:37.189812 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-4df48862-0ba2-4560-8353-72c1c995b9ff tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:38:37.190399 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-4df48862-0ba2-4560-8353-72c1c995b9ff tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:38:37.190828 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-4df48862-0ba2-4560-8353-72c1c995b9ff tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:38:37.214776 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-4df48862-0ba2-4560-8353-72c1c995b9ff tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] [instance: f5bad70c-350c-4230-a862-626dd6805e4c] Going to try to stop instance {{(pid=120864) force_stop /opt/stack/nova/nova/compute/api.py:2762}} Aug 27 02:38:37.263476 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-4df48862-0ba2-4560-8353-72c1c995b9ff tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] 10.222.0.52 "POST /compute/v2.1/servers/f5bad70c-350c-4230-a862-626dd6805e4c/action" status: 202 len: 0 microversion: 2.1 time: 0.108361 Aug 27 02:38:37.264182 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 551/1095] 10.222.0.52 () {60 vars in 1278 bytes} [Sat Aug 27 02:38:37 2022] POST /compute/v2.1/servers/f5bad70c-350c-4230-a862-626dd6805e4c/action => generated 0 bytes in 110 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Aug 27 02:38:37.274464 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-ad648bfe-85b2-41e5-9026-b5df8e8eb23d tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:38:37.275173 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-ad648bfe-85b2-41e5-9026-b5df8e8eb23d tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] [instance: f5bad70c-350c-4230-a862-626dd6805e4c] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:38:37.283896 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-ad648bfe-85b2-41e5-9026-b5df8e8eb23d tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:38:37.284960 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-ad648bfe-85b2-41e5-9026-b5df8e8eb23d tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:38:37.285544 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-ad648bfe-85b2-41e5-9026-b5df8e8eb23d tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:38:37.286494 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-ad648bfe-85b2-41e5-9026-b5df8e8eb23d tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:38:37.286972 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-ad648bfe-85b2-41e5-9026-b5df8e8eb23d tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:38:37.287393 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-ad648bfe-85b2-41e5-9026-b5df8e8eb23d tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:38:37.433275 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-ad648bfe-85b2-41e5-9026-b5df8e8eb23d tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee6084fffc544dfe8f1a9a30ae06be69', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a8055f4326f640fc8927f5cfdef39dd3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:38:37.443914 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-ad648bfe-85b2-41e5-9026-b5df8e8eb23d tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] 10.222.0.52 "GET /compute/v2.1/servers/f5bad70c-350c-4230-a862-626dd6805e4c" status: 200 len: 1676 microversion: 2.1 time: 0.174390 Aug 27 02:38:37.444873 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 545/1096] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:38:37 2022] GET /compute/v2.1/servers/f5bad70c-350c-4230-a862-626dd6805e4c => generated 1676 bytes in 176 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:38:38.476822 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-cebe5f49-f117-45b5-bfeb-bb231750408b tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Calling method '>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:38:38.477779 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-cebe5f49-f117-45b5-bfeb-bb231750408b tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] [instance: f5bad70c-350c-4230-a862-626dd6805e4c] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:38:38.487540 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-cebe5f49-f117-45b5-bfeb-bb231750408b tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:38:38.488191 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-cebe5f49-f117-45b5-bfeb-bb231750408b tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:38:38.488811 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-cebe5f49-f117-45b5-bfeb-bb231750408b tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:38:38.489803 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-cebe5f49-f117-45b5-bfeb-bb231750408b tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:38:38.490479 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-cebe5f49-f117-45b5-bfeb-bb231750408b tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:38:38.491044 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-cebe5f49-f117-45b5-bfeb-bb231750408b tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:38:38.679050 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-cebe5f49-f117-45b5-bfeb-bb231750408b tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee6084fffc544dfe8f1a9a30ae06be69', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a8055f4326f640fc8927f5cfdef39dd3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:38:38.687252 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-cebe5f49-f117-45b5-bfeb-bb231750408b tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] 10.222.0.52 "GET /compute/v2.1/servers/f5bad70c-350c-4230-a862-626dd6805e4c" status: 200 len: 1676 microversion: 2.1 time: 0.218396 Aug 27 02:38:38.688216 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 552/1097] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:38:38 2022] GET /compute/v2.1/servers/f5bad70c-350c-4230-a862-626dd6805e4c => generated 1676 bytes in 221 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:38:39.713258 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-90e2e81a-917d-458d-8de7-b5598bbc08b1 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:38:39.713948 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-90e2e81a-917d-458d-8de7-b5598bbc08b1 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] [instance: f5bad70c-350c-4230-a862-626dd6805e4c] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:38:39.721329 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-90e2e81a-917d-458d-8de7-b5598bbc08b1 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:38:39.721827 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-90e2e81a-917d-458d-8de7-b5598bbc08b1 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:38:39.722308 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-90e2e81a-917d-458d-8de7-b5598bbc08b1 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:38:39.723092 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-90e2e81a-917d-458d-8de7-b5598bbc08b1 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:38:39.723573 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-90e2e81a-917d-458d-8de7-b5598bbc08b1 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:38:39.724045 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-90e2e81a-917d-458d-8de7-b5598bbc08b1 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:38:39.870833 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-90e2e81a-917d-458d-8de7-b5598bbc08b1 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee6084fffc544dfe8f1a9a30ae06be69', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a8055f4326f640fc8927f5cfdef39dd3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:38:39.882689 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-90e2e81a-917d-458d-8de7-b5598bbc08b1 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] 10.222.0.52 "GET /compute/v2.1/servers/f5bad70c-350c-4230-a862-626dd6805e4c" status: 200 len: 1676 microversion: 2.1 time: 0.173777 Aug 27 02:38:39.883479 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 546/1098] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:38:39 2022] GET /compute/v2.1/servers/f5bad70c-350c-4230-a862-626dd6805e4c => generated 1676 bytes in 175 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:38:40.905268 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-7ffc2ff5-dffd-4ee7-95bc-e5bb721fec97 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Calling method '>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:38:40.906027 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-7ffc2ff5-dffd-4ee7-95bc-e5bb721fec97 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] [instance: f5bad70c-350c-4230-a862-626dd6805e4c] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:38:40.912994 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-7ffc2ff5-dffd-4ee7-95bc-e5bb721fec97 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:38:40.913514 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-7ffc2ff5-dffd-4ee7-95bc-e5bb721fec97 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:38:40.913945 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-7ffc2ff5-dffd-4ee7-95bc-e5bb721fec97 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:38:40.914642 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-7ffc2ff5-dffd-4ee7-95bc-e5bb721fec97 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:38:40.915105 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-7ffc2ff5-dffd-4ee7-95bc-e5bb721fec97 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:38:40.915491 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-7ffc2ff5-dffd-4ee7-95bc-e5bb721fec97 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:38:41.067623 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-7ffc2ff5-dffd-4ee7-95bc-e5bb721fec97 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee6084fffc544dfe8f1a9a30ae06be69', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a8055f4326f640fc8927f5cfdef39dd3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:38:41.079538 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-7ffc2ff5-dffd-4ee7-95bc-e5bb721fec97 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] 10.222.0.52 "GET /compute/v2.1/servers/f5bad70c-350c-4230-a862-626dd6805e4c" status: 200 len: 1676 microversion: 2.1 time: 0.177694 Aug 27 02:38:41.080353 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 553/1099] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:38:40 2022] GET /compute/v2.1/servers/f5bad70c-350c-4230-a862-626dd6805e4c => generated 1676 bytes in 180 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:38:42.104612 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-cfecbc4f-4ad4-43f7-98cc-fa702587f6ea tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:38:42.105260 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-cfecbc4f-4ad4-43f7-98cc-fa702587f6ea tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] [instance: f5bad70c-350c-4230-a862-626dd6805e4c] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:38:42.113157 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-cfecbc4f-4ad4-43f7-98cc-fa702587f6ea tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:38:42.113673 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-cfecbc4f-4ad4-43f7-98cc-fa702587f6ea tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:38:42.114117 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-cfecbc4f-4ad4-43f7-98cc-fa702587f6ea tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:38:42.114809 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-cfecbc4f-4ad4-43f7-98cc-fa702587f6ea tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:38:42.115261 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-cfecbc4f-4ad4-43f7-98cc-fa702587f6ea tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:38:42.115855 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-cfecbc4f-4ad4-43f7-98cc-fa702587f6ea tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:38:42.267802 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-cfecbc4f-4ad4-43f7-98cc-fa702587f6ea tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee6084fffc544dfe8f1a9a30ae06be69', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a8055f4326f640fc8927f5cfdef39dd3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:38:42.279052 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-cfecbc4f-4ad4-43f7-98cc-fa702587f6ea tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] 10.222.0.52 "GET /compute/v2.1/servers/f5bad70c-350c-4230-a862-626dd6805e4c" status: 200 len: 1676 microversion: 2.1 time: 0.178359 Aug 27 02:38:42.279642 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 547/1100] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:38:42 2022] GET /compute/v2.1/servers/f5bad70c-350c-4230-a862-626dd6805e4c => generated 1676 bytes in 180 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:38:43.302444 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-2e5fdf5a-72e5-4b03-abe7-35ac9fd09563 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Calling method '>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:38:43.303127 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-2e5fdf5a-72e5-4b03-abe7-35ac9fd09563 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] [instance: f5bad70c-350c-4230-a862-626dd6805e4c] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:38:43.311067 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-2e5fdf5a-72e5-4b03-abe7-35ac9fd09563 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:38:43.311563 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-2e5fdf5a-72e5-4b03-abe7-35ac9fd09563 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:38:43.311984 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-2e5fdf5a-72e5-4b03-abe7-35ac9fd09563 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:38:43.312685 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-2e5fdf5a-72e5-4b03-abe7-35ac9fd09563 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:38:43.313146 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-2e5fdf5a-72e5-4b03-abe7-35ac9fd09563 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:38:43.313608 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-2e5fdf5a-72e5-4b03-abe7-35ac9fd09563 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:38:43.455202 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-2e5fdf5a-72e5-4b03-abe7-35ac9fd09563 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee6084fffc544dfe8f1a9a30ae06be69', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a8055f4326f640fc8927f5cfdef39dd3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:38:43.463024 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-2e5fdf5a-72e5-4b03-abe7-35ac9fd09563 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] 10.222.0.52 "GET /compute/v2.1/servers/f5bad70c-350c-4230-a862-626dd6805e4c" status: 200 len: 1676 microversion: 2.1 time: 0.165522 Aug 27 02:38:43.463587 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 554/1101] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:38:43 2022] GET /compute/v2.1/servers/f5bad70c-350c-4230-a862-626dd6805e4c => generated 1676 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:38:44.490685 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-ff83989a-555d-44e7-be6b-61dd13f5c067 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:38:44.491559 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-ff83989a-555d-44e7-be6b-61dd13f5c067 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] [instance: f5bad70c-350c-4230-a862-626dd6805e4c] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:38:44.500938 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-ff83989a-555d-44e7-be6b-61dd13f5c067 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:38:44.501472 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-ff83989a-555d-44e7-be6b-61dd13f5c067 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:38:44.501867 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-ff83989a-555d-44e7-be6b-61dd13f5c067 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:38:44.502598 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-ff83989a-555d-44e7-be6b-61dd13f5c067 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:38:44.503060 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-ff83989a-555d-44e7-be6b-61dd13f5c067 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:38:44.503483 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-ff83989a-555d-44e7-be6b-61dd13f5c067 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:38:44.658303 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-ff83989a-555d-44e7-be6b-61dd13f5c067 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee6084fffc544dfe8f1a9a30ae06be69', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a8055f4326f640fc8927f5cfdef39dd3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:38:44.669378 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-ff83989a-555d-44e7-be6b-61dd13f5c067 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] 10.222.0.52 "GET /compute/v2.1/servers/f5bad70c-350c-4230-a862-626dd6805e4c" status: 200 len: 1676 microversion: 2.1 time: 0.187017 Aug 27 02:38:44.669991 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 548/1102] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:38:44 2022] GET /compute/v2.1/servers/f5bad70c-350c-4230-a862-626dd6805e4c => generated 1676 bytes in 189 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:38:45.696361 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-ef4adf99-2cd0-4191-a71f-9479a836e15d tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Calling method '>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:38:45.696930 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-ef4adf99-2cd0-4191-a71f-9479a836e15d tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] [instance: f5bad70c-350c-4230-a862-626dd6805e4c] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:38:45.704214 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-ef4adf99-2cd0-4191-a71f-9479a836e15d tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:38:45.704699 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-ef4adf99-2cd0-4191-a71f-9479a836e15d tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:38:45.705161 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-ef4adf99-2cd0-4191-a71f-9479a836e15d tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:38:45.705823 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-ef4adf99-2cd0-4191-a71f-9479a836e15d tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:38:45.706305 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-ef4adf99-2cd0-4191-a71f-9479a836e15d tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:38:45.706754 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-ef4adf99-2cd0-4191-a71f-9479a836e15d tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:38:46.003473 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-ef4adf99-2cd0-4191-a71f-9479a836e15d tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee6084fffc544dfe8f1a9a30ae06be69', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a8055f4326f640fc8927f5cfdef39dd3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:38:46.011388 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-ef4adf99-2cd0-4191-a71f-9479a836e15d tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] 10.222.0.52 "GET /compute/v2.1/servers/f5bad70c-350c-4230-a862-626dd6805e4c" status: 200 len: 1676 microversion: 2.1 time: 0.319108 Aug 27 02:38:46.011947 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 555/1103] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:38:45 2022] GET /compute/v2.1/servers/f5bad70c-350c-4230-a862-626dd6805e4c => generated 1676 bytes in 320 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:38:47.033931 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-ea8e9a3c-2758-4885-95e8-27635e5bd8d6 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:38:47.034513 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-ea8e9a3c-2758-4885-95e8-27635e5bd8d6 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] [instance: f5bad70c-350c-4230-a862-626dd6805e4c] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:38:47.041757 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-ea8e9a3c-2758-4885-95e8-27635e5bd8d6 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:38:47.042256 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-ea8e9a3c-2758-4885-95e8-27635e5bd8d6 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:38:47.042692 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-ea8e9a3c-2758-4885-95e8-27635e5bd8d6 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:38:47.043400 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-ea8e9a3c-2758-4885-95e8-27635e5bd8d6 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:38:47.043857 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-ea8e9a3c-2758-4885-95e8-27635e5bd8d6 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:38:47.044261 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-ea8e9a3c-2758-4885-95e8-27635e5bd8d6 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:38:47.181502 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-ea8e9a3c-2758-4885-95e8-27635e5bd8d6 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee6084fffc544dfe8f1a9a30ae06be69', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a8055f4326f640fc8927f5cfdef39dd3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:38:47.189943 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-ea8e9a3c-2758-4885-95e8-27635e5bd8d6 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] 10.222.0.52 "GET /compute/v2.1/servers/f5bad70c-350c-4230-a862-626dd6805e4c" status: 200 len: 1676 microversion: 2.1 time: 0.160147 Aug 27 02:38:47.190523 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 549/1104] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:38:47 2022] GET /compute/v2.1/servers/f5bad70c-350c-4230-a862-626dd6805e4c => generated 1676 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:38:48.215176 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-35369959-f91c-4c94-a1b3-753c012ef7db tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Calling method '>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:38:48.216288 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-35369959-f91c-4c94-a1b3-753c012ef7db tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] [instance: f5bad70c-350c-4230-a862-626dd6805e4c] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:38:48.223276 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-35369959-f91c-4c94-a1b3-753c012ef7db tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:38:48.223759 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-35369959-f91c-4c94-a1b3-753c012ef7db tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:38:48.224192 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-35369959-f91c-4c94-a1b3-753c012ef7db tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:38:48.224882 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-35369959-f91c-4c94-a1b3-753c012ef7db tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:38:48.225372 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-35369959-f91c-4c94-a1b3-753c012ef7db tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:38:48.225793 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-35369959-f91c-4c94-a1b3-753c012ef7db tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:38:48.379720 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-35369959-f91c-4c94-a1b3-753c012ef7db tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee6084fffc544dfe8f1a9a30ae06be69', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a8055f4326f640fc8927f5cfdef39dd3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:38:48.387777 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-35369959-f91c-4c94-a1b3-753c012ef7db tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] 10.222.0.52 "GET /compute/v2.1/servers/f5bad70c-350c-4230-a862-626dd6805e4c" status: 200 len: 1653 microversion: 2.1 time: 0.178492 Aug 27 02:38:48.388326 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 556/1105] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:38:48 2022] GET /compute/v2.1/servers/f5bad70c-350c-4230-a862-626dd6805e4c => generated 1653 bytes in 180 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:38:48.414131 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-0228fa90-0db4-4531-a9f8-d593d6abe96a tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Action: 'action', calling method: >, body: {"os-start": {}} {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 27 02:38:48.421490 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-0228fa90-0db4-4531-a9f8-d593d6abe96a tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:38:48.421992 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-0228fa90-0db4-4531-a9f8-d593d6abe96a tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:38:48.422434 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-0228fa90-0db4-4531-a9f8-d593d6abe96a tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:38:48.446357 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-0228fa90-0db4-4531-a9f8-d593d6abe96a tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] [instance: f5bad70c-350c-4230-a862-626dd6805e4c] Going to try to start instance {{(pid=120863) start /opt/stack/nova/nova/compute/api.py:2785}} Aug 27 02:38:48.499061 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-0228fa90-0db4-4531-a9f8-d593d6abe96a tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] 10.222.0.52 "POST /compute/v2.1/servers/f5bad70c-350c-4230-a862-626dd6805e4c/action" status: 202 len: 0 microversion: 2.1 time: 0.088677 Aug 27 02:38:48.499589 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 550/1106] 10.222.0.52 () {60 vars in 1278 bytes} [Sat Aug 27 02:38:48 2022] POST /compute/v2.1/servers/f5bad70c-350c-4230-a862-626dd6805e4c/action => generated 0 bytes in 90 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Aug 27 02:38:48.507617 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-e265f6bc-aa6e-49cb-b62e-e76737770cf3 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Calling method '>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:38:48.508240 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-e265f6bc-aa6e-49cb-b62e-e76737770cf3 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] [instance: f5bad70c-350c-4230-a862-626dd6805e4c] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:38:48.515716 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-e265f6bc-aa6e-49cb-b62e-e76737770cf3 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:38:48.516291 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-e265f6bc-aa6e-49cb-b62e-e76737770cf3 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:38:48.516760 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-e265f6bc-aa6e-49cb-b62e-e76737770cf3 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:38:48.517593 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-e265f6bc-aa6e-49cb-b62e-e76737770cf3 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:38:48.518076 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-e265f6bc-aa6e-49cb-b62e-e76737770cf3 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:38:48.518496 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-e265f6bc-aa6e-49cb-b62e-e76737770cf3 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:38:48.712568 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-e265f6bc-aa6e-49cb-b62e-e76737770cf3 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee6084fffc544dfe8f1a9a30ae06be69', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a8055f4326f640fc8927f5cfdef39dd3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:38:48.722137 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-e265f6bc-aa6e-49cb-b62e-e76737770cf3 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] 10.222.0.52 "GET /compute/v2.1/servers/f5bad70c-350c-4230-a862-626dd6805e4c" status: 200 len: 1662 microversion: 2.1 time: 0.218224 Aug 27 02:38:48.722725 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 557/1107] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:38:48 2022] GET /compute/v2.1/servers/f5bad70c-350c-4230-a862-626dd6805e4c => generated 1662 bytes in 219 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:38:49.746044 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-d4576cfd-b51b-4385-9ed1-0cedf44e10a7 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:38:49.746916 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-d4576cfd-b51b-4385-9ed1-0cedf44e10a7 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] [instance: f5bad70c-350c-4230-a862-626dd6805e4c] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:38:49.753520 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-d4576cfd-b51b-4385-9ed1-0cedf44e10a7 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:38:49.754018 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-d4576cfd-b51b-4385-9ed1-0cedf44e10a7 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:38:49.754469 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-d4576cfd-b51b-4385-9ed1-0cedf44e10a7 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:38:49.755227 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-d4576cfd-b51b-4385-9ed1-0cedf44e10a7 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:38:49.756216 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-d4576cfd-b51b-4385-9ed1-0cedf44e10a7 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:38:49.756216 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-d4576cfd-b51b-4385-9ed1-0cedf44e10a7 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:38:49.910147 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-d4576cfd-b51b-4385-9ed1-0cedf44e10a7 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee6084fffc544dfe8f1a9a30ae06be69', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a8055f4326f640fc8927f5cfdef39dd3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:38:49.918126 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-d4576cfd-b51b-4385-9ed1-0cedf44e10a7 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] 10.222.0.52 "GET /compute/v2.1/servers/f5bad70c-350c-4230-a862-626dd6805e4c" status: 200 len: 1662 microversion: 2.1 time: 0.177504 Aug 27 02:38:49.918719 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 551/1108] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:38:49 2022] GET /compute/v2.1/servers/f5bad70c-350c-4230-a862-626dd6805e4c => generated 1662 bytes in 179 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:38:50.457564 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-8bd07c15-d464-410d-b912-4fc254a2e21e tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Calling method '>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:38:50.458428 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-8bd07c15-d464-410d-b912-4fc254a2e21e tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] [instance: 30aa22c4-59b4-4482-b92a-3f386466f7ee] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:38:50.467857 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-8bd07c15-d464-410d-b912-4fc254a2e21e tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:38:50.468337 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-8bd07c15-d464-410d-b912-4fc254a2e21e tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:38:50.468767 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-8bd07c15-d464-410d-b912-4fc254a2e21e tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:38:50.469489 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-8bd07c15-d464-410d-b912-4fc254a2e21e tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:38:50.469942 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-8bd07c15-d464-410d-b912-4fc254a2e21e tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:38:50.470362 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-8bd07c15-d464-410d-b912-4fc254a2e21e tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:38:50.501028 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-8bd07c15-d464-410d-b912-4fc254a2e21e tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] [instance: 30aa22c4-59b4-4482-b92a-3f386466f7ee] Going to try to terminate instance {{(pid=120864) delete /opt/stack/nova/nova/compute/api.py:2719}} Aug 27 02:38:50.593448 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-8bd07c15-d464-410d-b912-4fc254a2e21e tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] 10.222.0.52 "DELETE /compute/v2.1/servers/30aa22c4-59b4-4482-b92a-3f386466f7ee" status: 204 len: 0 microversion: 2.1 time: 0.142673 Aug 27 02:38:50.594571 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 558/1109] 10.222.0.52 () {58 vars in 1239 bytes} [Sat Aug 27 02:38:50 2022] DELETE /compute/v2.1/servers/30aa22c4-59b4-4482-b92a-3f386466f7ee => generated 0 bytes in 145 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Aug 27 02:38:50.603214 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-edede310-f56c-41ac-b7bd-7d218a9c538b tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:38:50.603825 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-edede310-f56c-41ac-b7bd-7d218a9c538b tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] [instance: 30aa22c4-59b4-4482-b92a-3f386466f7ee] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:38:50.610338 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-edede310-f56c-41ac-b7bd-7d218a9c538b tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:38:50.610833 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-edede310-f56c-41ac-b7bd-7d218a9c538b tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:38:50.611269 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-edede310-f56c-41ac-b7bd-7d218a9c538b tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:38:50.611967 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-edede310-f56c-41ac-b7bd-7d218a9c538b tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:38:50.612420 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-edede310-f56c-41ac-b7bd-7d218a9c538b tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:38:50.612840 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-edede310-f56c-41ac-b7bd-7d218a9c538b tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:38:50.755409 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-edede310-f56c-41ac-b7bd-7d218a9c538b tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3fe5a1a037fa460eb3121e4d307542da', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '79e671941c234f36b542080d256de2c6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:38:50.763380 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-edede310-f56c-41ac-b7bd-7d218a9c538b tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] 10.222.0.52 "GET /compute/v2.1/servers/30aa22c4-59b4-4482-b92a-3f386466f7ee" status: 200 len: 1579 microversion: 2.1 time: 0.164405 Aug 27 02:38:50.763953 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 552/1110] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:38:50 2022] GET /compute/v2.1/servers/30aa22c4-59b4-4482-b92a-3f386466f7ee => generated 1579 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:38:50.941378 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-cd24e46a-6b11-4f0e-a1f9-8933ead33032 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Calling method '>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:38:50.942072 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-cd24e46a-6b11-4f0e-a1f9-8933ead33032 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] [instance: f5bad70c-350c-4230-a862-626dd6805e4c] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:38:50.949368 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-cd24e46a-6b11-4f0e-a1f9-8933ead33032 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:38:50.950008 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-cd24e46a-6b11-4f0e-a1f9-8933ead33032 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:38:50.950595 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-cd24e46a-6b11-4f0e-a1f9-8933ead33032 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:38:50.951266 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-cd24e46a-6b11-4f0e-a1f9-8933ead33032 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:38:50.951713 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-cd24e46a-6b11-4f0e-a1f9-8933ead33032 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:38:50.952130 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-cd24e46a-6b11-4f0e-a1f9-8933ead33032 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:38:51.107872 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-cd24e46a-6b11-4f0e-a1f9-8933ead33032 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee6084fffc544dfe8f1a9a30ae06be69', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a8055f4326f640fc8927f5cfdef39dd3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:38:51.116074 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-cd24e46a-6b11-4f0e-a1f9-8933ead33032 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] 10.222.0.52 "GET /compute/v2.1/servers/f5bad70c-350c-4230-a862-626dd6805e4c" status: 200 len: 1666 microversion: 2.1 time: 0.179188 Aug 27 02:38:51.116641 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 559/1111] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:38:50 2022] GET /compute/v2.1/servers/f5bad70c-350c-4230-a862-626dd6805e4c => generated 1666 bytes in 181 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:38:51.789132 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-1fcf334d-6b48-4b61-9cde-c57009bbda63 tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:38:51.789944 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-1fcf334d-6b48-4b61-9cde-c57009bbda63 tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] [instance: 30aa22c4-59b4-4482-b92a-3f386466f7ee] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:38:51.797817 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-1fcf334d-6b48-4b61-9cde-c57009bbda63 tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:38:51.798385 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-1fcf334d-6b48-4b61-9cde-c57009bbda63 tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:38:51.798931 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-1fcf334d-6b48-4b61-9cde-c57009bbda63 tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:38:51.799619 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-1fcf334d-6b48-4b61-9cde-c57009bbda63 tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:38:51.800088 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-1fcf334d-6b48-4b61-9cde-c57009bbda63 tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:38:51.800525 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-1fcf334d-6b48-4b61-9cde-c57009bbda63 tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:38:51.948097 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-1fcf334d-6b48-4b61-9cde-c57009bbda63 tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3fe5a1a037fa460eb3121e4d307542da', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '79e671941c234f36b542080d256de2c6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:38:51.957351 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-1fcf334d-6b48-4b61-9cde-c57009bbda63 tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] 10.222.0.52 "GET /compute/v2.1/servers/30aa22c4-59b4-4482-b92a-3f386466f7ee" status: 200 len: 1579 microversion: 2.1 time: 0.172888 Aug 27 02:38:51.957922 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 553/1112] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:38:51 2022] GET /compute/v2.1/servers/30aa22c4-59b4-4482-b92a-3f386466f7ee => generated 1579 bytes in 174 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:38:52.979967 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-6221cb08-c0de-4b41-9a26-642ebcd0c139 tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Calling method '>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:38:52.980556 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-6221cb08-c0de-4b41-9a26-642ebcd0c139 tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] [instance: 30aa22c4-59b4-4482-b92a-3f386466f7ee] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:38:52.987879 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-6221cb08-c0de-4b41-9a26-642ebcd0c139 tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:38:52.988509 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-6221cb08-c0de-4b41-9a26-642ebcd0c139 tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:38:52.988980 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-6221cb08-c0de-4b41-9a26-642ebcd0c139 tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:38:52.989710 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-6221cb08-c0de-4b41-9a26-642ebcd0c139 tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:38:52.990193 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-6221cb08-c0de-4b41-9a26-642ebcd0c139 tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:38:52.990623 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-6221cb08-c0de-4b41-9a26-642ebcd0c139 tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:38:53.080928 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-6221cb08-c0de-4b41-9a26-642ebcd0c139 tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3fe5a1a037fa460eb3121e4d307542da', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '79e671941c234f36b542080d256de2c6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:38:53.091576 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-6221cb08-c0de-4b41-9a26-642ebcd0c139 tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] 10.222.0.52 "GET /compute/v2.1/servers/30aa22c4-59b4-4482-b92a-3f386466f7ee" status: 200 len: 1512 microversion: 2.1 time: 0.114976 Aug 27 02:38:53.092455 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 560/1113] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:38:52 2022] GET /compute/v2.1/servers/30aa22c4-59b4-4482-b92a-3f386466f7ee => generated 1512 bytes in 117 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:38:53.373740 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [req-dff53350-ae10-4b3d-b5b8-4d1bada0c3cd req-c1812f63-e7c3-4878-b580-addc506f1aa6 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "30aa22c4-59b4-4482-b92a-3f386466f7ee", "name": "network-vif-deleted", "tag": "d3b3018c-6150-428a-a235-9ccd41ba80e7"}]} {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 27 02:38:53.385578 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [req-dff53350-ae10-4b3d-b5b8-4d1bada0c3cd req-c1812f63-e7c3-4878-b580-addc506f1aa6 service nova] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:38:53.386020 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [req-dff53350-ae10-4b3d-b5b8-4d1bada0c3cd req-c1812f63-e7c3-4878-b580-addc506f1aa6 service nova] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:38:53.386441 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [req-dff53350-ae10-4b3d-b5b8-4d1bada0c3cd req-c1812f63-e7c3-4878-b580-addc506f1aa6 service nova] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:38:53.398372 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.compute.server_external_events [req-dff53350-ae10-4b3d-b5b8-4d1bada0c3cd req-c1812f63-e7c3-4878-b580-addc506f1aa6 service nova] Creating event network-vif-deleted:d3b3018c-6150-428a-a235-9ccd41ba80e7 for instance 30aa22c4-59b4-4482-b92a-3f386466f7ee on hv-cinder-84630 Aug 27 02:38:53.409859 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [req-dff53350-ae10-4b3d-b5b8-4d1bada0c3cd req-c1812f63-e7c3-4878-b580-addc506f1aa6 service nova] 10.222.0.52 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.040190 Aug 27 02:38:53.410833 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 554/1114] 10.222.0.52 () {64 vars in 1317 bytes} [Sat Aug 27 02:38:53 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 42 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 27 02:38:54.112215 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-40118d7b-83f0-4a50-8e81-58adee730acc tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Calling method '>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:38:54.112978 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-40118d7b-83f0-4a50-8e81-58adee730acc tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] [instance: 30aa22c4-59b4-4482-b92a-3f386466f7ee] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:38:54.119888 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-40118d7b-83f0-4a50-8e81-58adee730acc tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:38:54.120387 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-40118d7b-83f0-4a50-8e81-58adee730acc tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:38:54.120814 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-40118d7b-83f0-4a50-8e81-58adee730acc tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:38:54.121560 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-40118d7b-83f0-4a50-8e81-58adee730acc tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:38:54.121989 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-40118d7b-83f0-4a50-8e81-58adee730acc tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:38:54.122415 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-40118d7b-83f0-4a50-8e81-58adee730acc tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:38:54.159864 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-40118d7b-83f0-4a50-8e81-58adee730acc tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:38:54.160358 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-40118d7b-83f0-4a50-8e81-58adee730acc tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:38:54.160795 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-40118d7b-83f0-4a50-8e81-58adee730acc tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:38:54.161304 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.objects.instance [None req-40118d7b-83f0-4a50-8e81-58adee730acc tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Lazy-loading 'fault' on Instance uuid 30aa22c4-59b4-4482-b92a-3f386466f7ee {{(pid=120864) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Aug 27 02:38:54.209525 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-40118d7b-83f0-4a50-8e81-58adee730acc tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3fe5a1a037fa460eb3121e4d307542da', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '79e671941c234f36b542080d256de2c6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:38:54.219689 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-40118d7b-83f0-4a50-8e81-58adee730acc tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] 10.222.0.52 "GET /compute/v2.1/servers/30aa22c4-59b4-4482-b92a-3f386466f7ee" status: 200 len: 1231 microversion: 2.1 time: 0.111293 Aug 27 02:38:54.220579 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 561/1115] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:38:54 2022] GET /compute/v2.1/servers/30aa22c4-59b4-4482-b92a-3f386466f7ee => generated 1231 bytes in 113 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:38:55.243775 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-f9281e67-272e-4e9d-9a86-88192fdb8eee tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:38:55.244346 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-f9281e67-272e-4e9d-9a86-88192fdb8eee tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] [instance: 30aa22c4-59b4-4482-b92a-3f386466f7ee] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:38:55.251555 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-f9281e67-272e-4e9d-9a86-88192fdb8eee tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:38:55.251987 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-f9281e67-272e-4e9d-9a86-88192fdb8eee tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:38:55.252417 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-f9281e67-272e-4e9d-9a86-88192fdb8eee tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:38:55.253138 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-f9281e67-272e-4e9d-9a86-88192fdb8eee tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:38:55.253566 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-f9281e67-272e-4e9d-9a86-88192fdb8eee tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:38:55.253982 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-f9281e67-272e-4e9d-9a86-88192fdb8eee tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:38:55.267988 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.wsgi [None req-f9281e67-272e-4e9d-9a86-88192fdb8eee tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] HTTP exception thrown: Instance 30aa22c4-59b4-4482-b92a-3f386466f7ee could not be found. Aug 27 02:38:55.269685 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-f9281e67-272e-4e9d-9a86-88192fdb8eee tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Request method failure captured: Aug 27 02:38:55.269685 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: request: GET /compute/v2.1/servers/30aa22c4-59b4-4482-b92a-3f386466f7ee HTTP/1.1 Aug 27 02:38:55.269685 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: Accept: application/json Aug 27 02:38:55.269685 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: Accept-Encoding: identity Aug 27 02:38:55.269685 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: Connection: close Aug 27 02:38:55.269685 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: Content-Length: 0 Aug 27 02:38:55.269685 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: Content-Type: application/json Aug 27 02:38:55.269685 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: Host: 10.222.0.52 Aug 27 02:38:55.269685 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: Openstack-System-Scope: None Aug 27 02:38:55.269685 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: User-Agent: python-urllib3/1.26.11 Aug 27 02:38:55.269685 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-Auth-Token: gAAAAABjCYLy85Uu3BiyLepTvyHrkionXI7siD38Rns3gISZZwndflckLvltMlTBHZJ6gk3GKZLgp1sOCr8IN2oWQwuE2aDeO0w7-eDMbQMQ90apHhdn2f8pHdjAco2Aqv2Vx1ThWF0nFT8Ht9XOqMF1ANor3utAF0ObK7gJJB5bN6WCMw2HLSw Aug 27 02:38:55.269685 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-Domain-Id: None Aug 27 02:38:55.269685 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-Domain-Name: None Aug 27 02:38:55.269685 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-Identity-Status: Confirmed Aug 27 02:38:55.269685 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-Is-Admin-Project: True Aug 27 02:38:55.269685 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-Project-Domain-Id: default Aug 27 02:38:55.269685 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-Project-Domain-Name: Default Aug 27 02:38:55.269685 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-Project-Id: 79e671941c234f36b542080d256de2c6 Aug 27 02:38:55.269685 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-Project-Name: tempest-TestVolumeBootPattern-1006469397 Aug 27 02:38:55.269685 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-Role: reader,member Aug 27 02:38:55.269685 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-Roles: reader,member Aug 27 02:38:55.271394 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-Service-Catalog: [{"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.52:8080", "publicURL": "http://10.222.0.52:8080/v1/AUTH_79e671941c234f36b542080d256de2c6"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52:9696/networking"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/compute/v2/79e671941c234f36b542080d256de2c6"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/image"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/volume/v3/79e671941c234f36b542080d256de2c6"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/identity"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/volume/v3/79e671941c234f36b542080d256de2c6"}]}] Aug 27 02:38:55.271394 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-Tenant: tempest-TestVolumeBootPattern-1006469397 Aug 27 02:38:55.271394 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-Tenant-Id: 79e671941c234f36b542080d256de2c6 Aug 27 02:38:55.271394 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-Tenant-Name: tempest-TestVolumeBootPattern-1006469397 Aug 27 02:38:55.271394 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-User: tempest-TestVolumeBootPattern-1006469397-project Aug 27 02:38:55.271394 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-User-Domain-Id: default Aug 27 02:38:55.271394 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-User-Domain-Name: Default Aug 27 02:38:55.271394 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-User-Id: 3fe5a1a037fa460eb3121e4d307542da Aug 27 02:38:55.271394 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-User-Name: tempest-TestVolumeBootPattern-1006469397-project Aug 27 02:38:55.271394 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: method: > Aug 27 02:38:55.271394 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: exception: Instance 30aa22c4-59b4-4482-b92a-3f386466f7ee could not be found. Aug 27 02:38:55.271394 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: action_args: {'id': '30aa22c4-59b4-4482-b92a-3f386466f7ee'} Aug 27 02:38:55.272487 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Aug 27 02:38:55.272487 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 27 02:38:55.272487 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Aug 27 02:38:55.272487 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Aug 27 02:38:55.272487 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Aug 27 02:38:55.272487 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Aug 27 02:38:55.272487 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Aug 27 02:38:55.272487 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Aug 27 02:38:55.272487 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Aug 27 02:38:55.272487 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Aug 27 02:38:55.272487 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 30aa22c4-59b4-4482-b92a-3f386466f7ee could not be found. Aug 27 02:38:55.272487 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi Aug 27 02:38:55.272487 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 27 02:38:55.272487 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi Aug 27 02:38:55.272487 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 27 02:38:55.272487 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 27 02:38:55.272487 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 27 02:38:55.272487 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Aug 27 02:38:55.272487 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Aug 27 02:38:55.272487 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Aug 27 02:38:55.273901 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Aug 27 02:38:55.273901 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Aug 27 02:38:55.273901 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi instance = self._get_server( Aug 27 02:38:55.273901 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Aug 27 02:38:55.273901 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Aug 27 02:38:55.273901 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Aug 27 02:38:55.273901 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Aug 27 02:38:55.273901 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 30aa22c4-59b4-4482-b92a-3f386466f7ee could not be found. Aug 27 02:38:55.273901 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi Aug 27 02:38:55.273901 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 27 02:38:55.273901 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi Aug 27 02:38:55.273901 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 27 02:38:55.273901 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 27 02:38:55.273901 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 27 02:38:55.273901 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Aug 27 02:38:55.273901 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Aug 27 02:38:55.273901 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 30aa22c4-59b4-4482-b92a-3f386466f7ee could not be found. Aug 27 02:38:55.273901 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi Aug 27 02:38:55.275109 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-f9281e67-272e-4e9d-9a86-88192fdb8eee tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Returning 404 to user: Instance 30aa22c4-59b4-4482-b92a-3f386466f7ee could not be found. {{(pid=120863) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Aug 27 02:38:55.275109 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-f9281e67-272e-4e9d-9a86-88192fdb8eee tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] 10.222.0.52 "GET /compute/v2.1/servers/30aa22c4-59b4-4482-b92a-3f386466f7ee" status: 404 len: 111 microversion: 2.1 time: 0.032392 Aug 27 02:38:55.275109 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 555/1116] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:38:55 2022] GET /compute/v2.1/servers/30aa22c4-59b4-4482-b92a-3f386466f7ee => generated 111 bytes in 34 msecs (HTTP/1.1 404) 9 headers in 379 bytes (2 switches on core 0) Aug 27 02:38:55.322139 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-fb89d11d-3f1b-4316-87c4-0f84adfb5465 tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-TestVolumeBootPattern-server-1725994313", "imageRef": "", "flavorRef": "452", "key_name": "tempest-TestVolumeBootPattern-1366561407", "security_groups": [{"name": "tempest-secgroup-smoke-743539963"}], "block_device_mapping_v2": [{"uuid": "8509b4bf-c8cd-41be-817e-d010c8fe3bdd", "source_type": "volume", "destination_type": "volume", "boot_index": 0, "delete_on_termination": false}], "networks": [{"uuid": "3a1ac093-e4b6-48ea-aff8-2575f231ba35"}]}} {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 27 02:38:55.446049 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.network.neutron [None req-fb89d11d-3f1b-4316-87c4-0f84adfb5465 tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] validate_networks() for [('3a1ac093-e4b6-48ea-aff8-2575f231ba35', None, None, None, None, None)] {{(pid=120864) validate_networks /opt/stack/nova/nova/network/neutron.py:2626}} Aug 27 02:38:55.868029 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.virt.hardware [None req-fb89d11d-3f1b-4316-87c4-0f84adfb5465 tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Flavor limits 0:0:0 {{(pid=120864) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Aug 27 02:38:55.868453 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.virt.hardware [None req-fb89d11d-3f1b-4316-87c4-0f84adfb5465 tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Image limits 0:0:0 {{(pid=120864) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Aug 27 02:38:55.868926 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.virt.hardware [None req-fb89d11d-3f1b-4316-87c4-0f84adfb5465 tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Flavor pref 0:0:0 {{(pid=120864) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Aug 27 02:38:55.869348 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.virt.hardware [None req-fb89d11d-3f1b-4316-87c4-0f84adfb5465 tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Image pref 0:0:0 {{(pid=120864) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Aug 27 02:38:55.869787 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.virt.hardware [None req-fb89d11d-3f1b-4316-87c4-0f84adfb5465 tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=120864) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Aug 27 02:38:55.881470 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-fb89d11d-3f1b-4316-87c4-0f84adfb5465 tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:38:55.881959 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-fb89d11d-3f1b-4316-87c4-0f84adfb5465 tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:38:55.882387 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-fb89d11d-3f1b-4316-87c4-0f84adfb5465 tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:38:55.883119 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-fb89d11d-3f1b-4316-87c4-0f84adfb5465 tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:38:55.883570 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-fb89d11d-3f1b-4316-87c4-0f84adfb5465 tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:38:55.883996 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-fb89d11d-3f1b-4316-87c4-0f84adfb5465 tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:38:55.903005 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.quota [None req-fb89d11d-3f1b-4316-87c4-0f84adfb5465 tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Getting quotas for project 79e671941c234f36b542080d256de2c6. Resources: {'cores', 'ram', 'instances'} {{(pid=120864) _get_quotas /opt/stack/nova/nova/quota.py:393}} Aug 27 02:38:55.909059 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.quota [None req-fb89d11d-3f1b-4316-87c4-0f84adfb5465 tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Getting quotas for user 3fe5a1a037fa460eb3121e4d307542da and project 79e671941c234f36b542080d256de2c6. Resources: {'cores', 'ram', 'instances'} {{(pid=120864) _get_quotas /opt/stack/nova/nova/quota.py:383}} Aug 27 02:38:55.973967 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-fb89d11d-3f1b-4316-87c4-0f84adfb5465 tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Going to run 1 instances... {{(pid=120864) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Aug 27 02:38:55.976076 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-fb89d11d-3f1b-4316-87c4-0f84adfb5465 tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] [instance: 2afc317e-fd8d-4e86-89d9-aba4b76d83cb] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=False,deleted=,deleted_at=,destination_type='volume',device_name=None,device_type=None,disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id=None,instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='volume',tag=None,updated_at=,uuid=,volume_id='8509b4bf-c8cd-41be-817e-d010c8fe3bdd',volume_size=None,volume_type=None)] {{(pid=120864) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Aug 27 02:38:56.117532 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-fb89d11d-3f1b-4316-87c4-0f84adfb5465 tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] 10.222.0.52 "POST /compute/v2.1/servers" status: 202 len: 405 microversion: 2.1 time: 0.799117 Aug 27 02:38:56.118102 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 562/1117] 10.222.0.52 () {60 vars in 1147 bytes} [Sat Aug 27 02:38:55 2022] POST /compute/v2.1/servers => generated 405 bytes in 800 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Aug 27 02:38:56.129428 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-884bf05d-431d-47a8-bc15-841dfd75e247 tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:38:56.130042 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-884bf05d-431d-47a8-bc15-841dfd75e247 tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] [instance: 2afc317e-fd8d-4e86-89d9-aba4b76d83cb] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:38:56.186303 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-884bf05d-431d-47a8-bc15-841dfd75e247 tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3fe5a1a037fa460eb3121e4d307542da', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '79e671941c234f36b542080d256de2c6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:38:56.195997 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-884bf05d-431d-47a8-bc15-841dfd75e247 tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] 10.222.0.52 "GET /compute/v2.1/servers/2afc317e-fd8d-4e86-89d9-aba4b76d83cb" status: 200 len: 1095 microversion: 2.1 time: 0.070229 Aug 27 02:38:56.196583 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 556/1118] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:38:56 2022] GET /compute/v2.1/servers/2afc317e-fd8d-4e86-89d9-aba4b76d83cb => generated 1095 bytes in 72 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:38:57.218648 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-4bc86031-353a-4a5d-af16-c46925fbb2d1 tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Calling method '>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:38:57.219215 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-4bc86031-353a-4a5d-af16-c46925fbb2d1 tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] [instance: 2afc317e-fd8d-4e86-89d9-aba4b76d83cb] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:38:57.225573 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-4bc86031-353a-4a5d-af16-c46925fbb2d1 tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:38:57.226062 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-4bc86031-353a-4a5d-af16-c46925fbb2d1 tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:38:57.226497 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-4bc86031-353a-4a5d-af16-c46925fbb2d1 tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:38:57.227194 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-4bc86031-353a-4a5d-af16-c46925fbb2d1 tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:38:57.227645 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-4bc86031-353a-4a5d-af16-c46925fbb2d1 tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:38:57.228104 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-4bc86031-353a-4a5d-af16-c46925fbb2d1 tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:38:57.326375 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-4bc86031-353a-4a5d-af16-c46925fbb2d1 tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3fe5a1a037fa460eb3121e4d307542da', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '79e671941c234f36b542080d256de2c6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:38:57.335527 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-4bc86031-353a-4a5d-af16-c46925fbb2d1 tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] 10.222.0.52 "GET /compute/v2.1/servers/2afc317e-fd8d-4e86-89d9-aba4b76d83cb" status: 200 len: 1201 microversion: 2.1 time: 0.121511 Aug 27 02:38:57.336229 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 563/1119] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:38:57 2022] GET /compute/v2.1/servers/2afc317e-fd8d-4e86-89d9-aba4b76d83cb => generated 1201 bytes in 123 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:38:58.357179 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-aceed772-aa6f-4be3-aa58-ffff061e9811 tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:38:58.357830 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-aceed772-aa6f-4be3-aa58-ffff061e9811 tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] [instance: 2afc317e-fd8d-4e86-89d9-aba4b76d83cb] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:38:58.365456 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-aceed772-aa6f-4be3-aa58-ffff061e9811 tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:38:58.365974 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-aceed772-aa6f-4be3-aa58-ffff061e9811 tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:38:58.366408 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-aceed772-aa6f-4be3-aa58-ffff061e9811 tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:38:58.367130 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-aceed772-aa6f-4be3-aa58-ffff061e9811 tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:38:58.367612 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-aceed772-aa6f-4be3-aa58-ffff061e9811 tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:38:58.368034 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-aceed772-aa6f-4be3-aa58-ffff061e9811 tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:38:58.581330 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-aceed772-aa6f-4be3-aa58-ffff061e9811 tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3fe5a1a037fa460eb3121e4d307542da', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '79e671941c234f36b542080d256de2c6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:38:58.590417 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-aceed772-aa6f-4be3-aa58-ffff061e9811 tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] 10.222.0.52 "GET /compute/v2.1/servers/2afc317e-fd8d-4e86-89d9-aba4b76d83cb" status: 200 len: 1266 microversion: 2.1 time: 0.237136 Aug 27 02:38:58.591341 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 557/1120] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:38:58 2022] GET /compute/v2.1/servers/2afc317e-fd8d-4e86-89d9-aba4b76d83cb => generated 1266 bytes in 239 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:38:58.806642 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [req-ef3da1ef-5fad-4711-a222-52e5cef03b16 req-ad099d9d-4c23-4ace-ac44-ab3aa9edc244 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "2afc317e-fd8d-4e86-89d9-aba4b76d83cb", "tag": "df547cda-45a3-4fd8-bd2f-f3b5e1938759"}]} {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 27 02:38:58.818908 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [req-ef3da1ef-5fad-4711-a222-52e5cef03b16 req-ad099d9d-4c23-4ace-ac44-ab3aa9edc244 service nova] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:38:58.819389 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [req-ef3da1ef-5fad-4711-a222-52e5cef03b16 req-ad099d9d-4c23-4ace-ac44-ab3aa9edc244 service nova] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:38:58.819798 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [req-ef3da1ef-5fad-4711-a222-52e5cef03b16 req-ad099d9d-4c23-4ace-ac44-ab3aa9edc244 service nova] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:38:58.832767 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.compute.server_external_events [req-ef3da1ef-5fad-4711-a222-52e5cef03b16 req-ad099d9d-4c23-4ace-ac44-ab3aa9edc244 service nova] Creating event network-changed:df547cda-45a3-4fd8-bd2f-f3b5e1938759 for instance 2afc317e-fd8d-4e86-89d9-aba4b76d83cb on hv-cinder-84630 Aug 27 02:38:58.837932 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [req-ef3da1ef-5fad-4711-a222-52e5cef03b16 req-ad099d9d-4c23-4ace-ac44-ab3aa9edc244 service nova] 10.222.0.52 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.035417 Aug 27 02:38:58.838518 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 564/1121] 10.222.0.52 () {64 vars in 1317 bytes} [Sat Aug 27 02:38:58 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 37 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Aug 27 02:38:59.611131 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-cfd8ffb5-6945-44b4-ac4c-84c6335cc28c tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:38:59.611711 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-cfd8ffb5-6945-44b4-ac4c-84c6335cc28c tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] [instance: 2afc317e-fd8d-4e86-89d9-aba4b76d83cb] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:38:59.618575 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-cfd8ffb5-6945-44b4-ac4c-84c6335cc28c tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:38:59.619061 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-cfd8ffb5-6945-44b4-ac4c-84c6335cc28c tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:38:59.619498 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-cfd8ffb5-6945-44b4-ac4c-84c6335cc28c tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:38:59.620217 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-cfd8ffb5-6945-44b4-ac4c-84c6335cc28c tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:38:59.621140 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-cfd8ffb5-6945-44b4-ac4c-84c6335cc28c tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:38:59.621827 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-cfd8ffb5-6945-44b4-ac4c-84c6335cc28c tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:38:59.758845 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-cfd8ffb5-6945-44b4-ac4c-84c6335cc28c tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3fe5a1a037fa460eb3121e4d307542da', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '79e671941c234f36b542080d256de2c6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:38:59.766845 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-cfd8ffb5-6945-44b4-ac4c-84c6335cc28c tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] 10.222.0.52 "GET /compute/v2.1/servers/2afc317e-fd8d-4e86-89d9-aba4b76d83cb" status: 200 len: 1266 microversion: 2.1 time: 0.159647 Aug 27 02:38:59.767414 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 558/1122] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:38:59 2022] GET /compute/v2.1/servers/2afc317e-fd8d-4e86-89d9-aba4b76d83cb => generated 1266 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:39:00.791758 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-8792ddf8-1628-4e72-bc63-a1011814c84c tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Calling method '>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:39:00.792517 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-8792ddf8-1628-4e72-bc63-a1011814c84c tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] [instance: 2afc317e-fd8d-4e86-89d9-aba4b76d83cb] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:39:00.799945 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-8792ddf8-1628-4e72-bc63-a1011814c84c tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:39:00.800424 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-8792ddf8-1628-4e72-bc63-a1011814c84c tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:39:00.800850 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-8792ddf8-1628-4e72-bc63-a1011814c84c tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:39:00.801575 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-8792ddf8-1628-4e72-bc63-a1011814c84c tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:39:00.802021 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-8792ddf8-1628-4e72-bc63-a1011814c84c tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:39:00.802437 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-8792ddf8-1628-4e72-bc63-a1011814c84c tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:39:00.942267 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-8792ddf8-1628-4e72-bc63-a1011814c84c tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3fe5a1a037fa460eb3121e4d307542da', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '79e671941c234f36b542080d256de2c6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:39:00.945941 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG dbcounter [-] [120864] Writing DB stats nova_cell1:SELECT=96,nova_cell1:UPDATE=2,nova_cell1:INSERT=2 {{(pid=120864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 27 02:39:00.951050 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-8792ddf8-1628-4e72-bc63-a1011814c84c tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] 10.222.0.52 "GET /compute/v2.1/servers/2afc317e-fd8d-4e86-89d9-aba4b76d83cb" status: 200 len: 1266 microversion: 2.1 time: 0.165622 Aug 27 02:39:00.951929 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 565/1123] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:39:00 2022] GET /compute/v2.1/servers/2afc317e-fd8d-4e86-89d9-aba4b76d83cb => generated 1266 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:39:01.976047 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-b2aa6268-4e12-4bc9-aaa8-1c2ea3582f57 tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:39:01.977347 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-b2aa6268-4e12-4bc9-aaa8-1c2ea3582f57 tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] [instance: 2afc317e-fd8d-4e86-89d9-aba4b76d83cb] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:39:01.984834 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-b2aa6268-4e12-4bc9-aaa8-1c2ea3582f57 tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:39:01.985491 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-b2aa6268-4e12-4bc9-aaa8-1c2ea3582f57 tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:39:01.986090 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-b2aa6268-4e12-4bc9-aaa8-1c2ea3582f57 tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:39:01.987138 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-b2aa6268-4e12-4bc9-aaa8-1c2ea3582f57 tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:39:01.987732 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-b2aa6268-4e12-4bc9-aaa8-1c2ea3582f57 tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:39:01.988254 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-b2aa6268-4e12-4bc9-aaa8-1c2ea3582f57 tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:39:02.134726 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-b2aa6268-4e12-4bc9-aaa8-1c2ea3582f57 tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3fe5a1a037fa460eb3121e4d307542da', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '79e671941c234f36b542080d256de2c6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:39:02.142977 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-b2aa6268-4e12-4bc9-aaa8-1c2ea3582f57 tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] 10.222.0.52 "GET /compute/v2.1/servers/2afc317e-fd8d-4e86-89d9-aba4b76d83cb" status: 200 len: 1266 microversion: 2.1 time: 0.172490 Aug 27 02:39:02.143551 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 559/1124] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:39:01 2022] GET /compute/v2.1/servers/2afc317e-fd8d-4e86-89d9-aba4b76d83cb => generated 1266 bytes in 174 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:39:03.164675 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-f2832577-4509-423e-9a9c-7bec19242f59 tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Calling method '>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:39:03.165359 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-f2832577-4509-423e-9a9c-7bec19242f59 tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] [instance: 2afc317e-fd8d-4e86-89d9-aba4b76d83cb] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:39:03.173907 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-f2832577-4509-423e-9a9c-7bec19242f59 tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:39:03.174400 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-f2832577-4509-423e-9a9c-7bec19242f59 tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:39:03.174833 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-f2832577-4509-423e-9a9c-7bec19242f59 tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:39:03.175533 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-f2832577-4509-423e-9a9c-7bec19242f59 tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:39:03.175984 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-f2832577-4509-423e-9a9c-7bec19242f59 tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:39:03.176393 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-f2832577-4509-423e-9a9c-7bec19242f59 tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:39:03.315840 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-f2832577-4509-423e-9a9c-7bec19242f59 tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3fe5a1a037fa460eb3121e4d307542da', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '79e671941c234f36b542080d256de2c6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:39:03.323931 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-f2832577-4509-423e-9a9c-7bec19242f59 tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] 10.222.0.52 "GET /compute/v2.1/servers/2afc317e-fd8d-4e86-89d9-aba4b76d83cb" status: 200 len: 1266 microversion: 2.1 time: 0.163161 Aug 27 02:39:03.324483 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 566/1125] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:39:03 2022] GET /compute/v2.1/servers/2afc317e-fd8d-4e86-89d9-aba4b76d83cb => generated 1266 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:39:04.345430 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-4310ee1c-e597-4eea-a4fc-329d26f53375 tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:39:04.346081 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-4310ee1c-e597-4eea-a4fc-329d26f53375 tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] [instance: 2afc317e-fd8d-4e86-89d9-aba4b76d83cb] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:39:04.352818 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-4310ee1c-e597-4eea-a4fc-329d26f53375 tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:39:04.353346 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-4310ee1c-e597-4eea-a4fc-329d26f53375 tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:39:04.353783 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-4310ee1c-e597-4eea-a4fc-329d26f53375 tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:39:04.354651 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-4310ee1c-e597-4eea-a4fc-329d26f53375 tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:39:04.355316 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-4310ee1c-e597-4eea-a4fc-329d26f53375 tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:39:04.355753 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-4310ee1c-e597-4eea-a4fc-329d26f53375 tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:39:04.750532 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-4310ee1c-e597-4eea-a4fc-329d26f53375 tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3fe5a1a037fa460eb3121e4d307542da', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '79e671941c234f36b542080d256de2c6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:39:04.760325 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-4310ee1c-e597-4eea-a4fc-329d26f53375 tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] 10.222.0.52 "GET /compute/v2.1/servers/2afc317e-fd8d-4e86-89d9-aba4b76d83cb" status: 200 len: 1266 microversion: 2.1 time: 0.418848 Aug 27 02:39:04.761008 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 560/1126] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:39:04 2022] GET /compute/v2.1/servers/2afc317e-fd8d-4e86-89d9-aba4b76d83cb => generated 1266 bytes in 420 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:39:05.789693 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-4634cbec-455b-4772-aa8c-226dd58fc8ae tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Calling method '>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:39:05.790551 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-4634cbec-455b-4772-aa8c-226dd58fc8ae tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] [instance: 2afc317e-fd8d-4e86-89d9-aba4b76d83cb] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:39:05.797975 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-4634cbec-455b-4772-aa8c-226dd58fc8ae tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:39:05.798461 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-4634cbec-455b-4772-aa8c-226dd58fc8ae tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:39:05.798928 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-4634cbec-455b-4772-aa8c-226dd58fc8ae tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:39:05.799630 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-4634cbec-455b-4772-aa8c-226dd58fc8ae tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:39:05.800081 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-4634cbec-455b-4772-aa8c-226dd58fc8ae tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:39:05.800479 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-4634cbec-455b-4772-aa8c-226dd58fc8ae tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:39:05.891104 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG dbcounter [-] [120864] Writing DB stats nova_cell0:SELECT=2 {{(pid=120864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 27 02:39:05.916851 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG dbcounter [-] [120864] Writing DB stats nova_cell0:SELECT=6 {{(pid=120864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 27 02:39:05.939251 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-4634cbec-455b-4772-aa8c-226dd58fc8ae tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3fe5a1a037fa460eb3121e4d307542da', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '79e671941c234f36b542080d256de2c6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:39:05.950039 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-4634cbec-455b-4772-aa8c-226dd58fc8ae tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] 10.222.0.52 "GET /compute/v2.1/servers/2afc317e-fd8d-4e86-89d9-aba4b76d83cb" status: 200 len: 1266 microversion: 2.1 time: 0.166805 Aug 27 02:39:05.950609 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 567/1127] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:39:05 2022] GET /compute/v2.1/servers/2afc317e-fd8d-4e86-89d9-aba4b76d83cb => generated 1266 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:39:06.972118 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-ed587e76-540c-4cc7-9466-3a290e6e582c tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:39:06.972733 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-ed587e76-540c-4cc7-9466-3a290e6e582c tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] [instance: 2afc317e-fd8d-4e86-89d9-aba4b76d83cb] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:39:06.975115 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG dbcounter [-] [120863] Writing DB stats nova_cell0:SELECT=2 {{(pid=120863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 27 02:39:06.975751 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG dbcounter [-] [120863] Writing DB stats nova_api:SELECT=37 {{(pid=120863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 27 02:39:06.981853 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-ed587e76-540c-4cc7-9466-3a290e6e582c tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:39:06.982331 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-ed587e76-540c-4cc7-9466-3a290e6e582c tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:39:06.982763 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-ed587e76-540c-4cc7-9466-3a290e6e582c tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:39:06.983527 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-ed587e76-540c-4cc7-9466-3a290e6e582c tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:39:06.983976 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-ed587e76-540c-4cc7-9466-3a290e6e582c tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:39:06.984450 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-ed587e76-540c-4cc7-9466-3a290e6e582c tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:39:06.989012 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG dbcounter [-] [120863] Writing DB stats nova_cell1:SELECT=96,nova_cell1:INSERT=2,nova_cell1:UPDATE=1 {{(pid=120863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 27 02:39:07.138219 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-ed587e76-540c-4cc7-9466-3a290e6e582c tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3fe5a1a037fa460eb3121e4d307542da', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '79e671941c234f36b542080d256de2c6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:39:07.147669 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-ed587e76-540c-4cc7-9466-3a290e6e582c tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] 10.222.0.52 "GET /compute/v2.1/servers/2afc317e-fd8d-4e86-89d9-aba4b76d83cb" status: 200 len: 1266 microversion: 2.1 time: 0.179895 Aug 27 02:39:07.148243 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 561/1128] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:39:06 2022] GET /compute/v2.1/servers/2afc317e-fd8d-4e86-89d9-aba4b76d83cb => generated 1266 bytes in 181 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:39:08.174880 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-1d7328d8-7893-4f9b-b105-1bf9c1d60ac3 tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Calling method '>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:39:08.175501 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-1d7328d8-7893-4f9b-b105-1bf9c1d60ac3 tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] [instance: 2afc317e-fd8d-4e86-89d9-aba4b76d83cb] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:39:08.177358 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG dbcounter [-] [120864] Writing DB stats nova_api:SELECT=55,nova_api:UPDATE=1,nova_api:INSERT=3 {{(pid=120864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 27 02:39:08.186113 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-1d7328d8-7893-4f9b-b105-1bf9c1d60ac3 tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:39:08.186639 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-1d7328d8-7893-4f9b-b105-1bf9c1d60ac3 tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:39:08.187128 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-1d7328d8-7893-4f9b-b105-1bf9c1d60ac3 tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:39:08.187812 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-1d7328d8-7893-4f9b-b105-1bf9c1d60ac3 tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:39:08.188276 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-1d7328d8-7893-4f9b-b105-1bf9c1d60ac3 tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:39:08.188680 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-1d7328d8-7893-4f9b-b105-1bf9c1d60ac3 tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:39:08.329105 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-1d7328d8-7893-4f9b-b105-1bf9c1d60ac3 tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3fe5a1a037fa460eb3121e4d307542da', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '79e671941c234f36b542080d256de2c6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:39:08.337150 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-1d7328d8-7893-4f9b-b105-1bf9c1d60ac3 tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] 10.222.0.52 "GET /compute/v2.1/servers/2afc317e-fd8d-4e86-89d9-aba4b76d83cb" status: 200 len: 1266 microversion: 2.1 time: 0.166604 Aug 27 02:39:08.338062 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 568/1129] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:39:08 2022] GET /compute/v2.1/servers/2afc317e-fd8d-4e86-89d9-aba4b76d83cb => generated 1266 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:39:09.360146 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-68acb6b3-9a2b-4197-b01e-811091beee06 tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:39:09.360817 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-68acb6b3-9a2b-4197-b01e-811091beee06 tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] [instance: 2afc317e-fd8d-4e86-89d9-aba4b76d83cb] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:39:09.367572 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-68acb6b3-9a2b-4197-b01e-811091beee06 tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:39:09.368060 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-68acb6b3-9a2b-4197-b01e-811091beee06 tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:39:09.368493 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-68acb6b3-9a2b-4197-b01e-811091beee06 tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:39:09.369296 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-68acb6b3-9a2b-4197-b01e-811091beee06 tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:39:09.369758 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-68acb6b3-9a2b-4197-b01e-811091beee06 tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:39:09.370341 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-68acb6b3-9a2b-4197-b01e-811091beee06 tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:39:09.519865 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-68acb6b3-9a2b-4197-b01e-811091beee06 tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3fe5a1a037fa460eb3121e4d307542da', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '79e671941c234f36b542080d256de2c6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:39:09.528426 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-68acb6b3-9a2b-4197-b01e-811091beee06 tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] 10.222.0.52 "GET /compute/v2.1/servers/2afc317e-fd8d-4e86-89d9-aba4b76d83cb" status: 200 len: 1266 microversion: 2.1 time: 0.172272 Aug 27 02:39:09.529004 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 562/1130] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:39:09 2022] GET /compute/v2.1/servers/2afc317e-fd8d-4e86-89d9-aba4b76d83cb => generated 1266 bytes in 174 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:39:09.878319 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [req-98d1f884-e9cc-44b8-86bb-4c612fd4f163 req-b0eb0df8-98e7-4fb1-8a46-518817c7830a service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "2afc317e-fd8d-4e86-89d9-aba4b76d83cb", "name": "network-vif-plugged", "status": "completed", "tag": "df547cda-45a3-4fd8-bd2f-f3b5e1938759"}]} {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 27 02:39:09.890453 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [req-98d1f884-e9cc-44b8-86bb-4c612fd4f163 req-b0eb0df8-98e7-4fb1-8a46-518817c7830a service nova] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:39:09.890926 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [req-98d1f884-e9cc-44b8-86bb-4c612fd4f163 req-b0eb0df8-98e7-4fb1-8a46-518817c7830a service nova] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:39:09.891349 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [req-98d1f884-e9cc-44b8-86bb-4c612fd4f163 req-b0eb0df8-98e7-4fb1-8a46-518817c7830a service nova] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:39:09.903947 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.compute.server_external_events [req-98d1f884-e9cc-44b8-86bb-4c612fd4f163 req-b0eb0df8-98e7-4fb1-8a46-518817c7830a service nova] Creating event network-vif-plugged:df547cda-45a3-4fd8-bd2f-f3b5e1938759 for instance 2afc317e-fd8d-4e86-89d9-aba4b76d83cb on hv-cinder-84630 Aug 27 02:39:09.910046 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [req-98d1f884-e9cc-44b8-86bb-4c612fd4f163 req-b0eb0df8-98e7-4fb1-8a46-518817c7830a service nova] 10.222.0.52 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.120138 Aug 27 02:39:09.910917 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 569/1131] 10.222.0.52 () {64 vars in 1317 bytes} [Sat Aug 27 02:39:09 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 122 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 27 02:39:10.553819 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-4a31e76c-aa3c-4d1f-92db-b9564595fe4c tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:39:10.554697 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-4a31e76c-aa3c-4d1f-92db-b9564595fe4c tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] [instance: 2afc317e-fd8d-4e86-89d9-aba4b76d83cb] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:39:10.564106 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-4a31e76c-aa3c-4d1f-92db-b9564595fe4c tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:39:10.564779 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-4a31e76c-aa3c-4d1f-92db-b9564595fe4c tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:39:10.565359 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-4a31e76c-aa3c-4d1f-92db-b9564595fe4c tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:39:10.566227 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-4a31e76c-aa3c-4d1f-92db-b9564595fe4c tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:39:10.566841 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-4a31e76c-aa3c-4d1f-92db-b9564595fe4c tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:39:10.567261 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-4a31e76c-aa3c-4d1f-92db-b9564595fe4c tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:39:10.721463 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-4a31e76c-aa3c-4d1f-92db-b9564595fe4c tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3fe5a1a037fa460eb3121e4d307542da', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '79e671941c234f36b542080d256de2c6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:39:10.732797 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-4a31e76c-aa3c-4d1f-92db-b9564595fe4c tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] 10.222.0.52 "GET /compute/v2.1/servers/2afc317e-fd8d-4e86-89d9-aba4b76d83cb" status: 200 len: 1266 microversion: 2.1 time: 0.185535 Aug 27 02:39:10.733673 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 563/1132] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:39:10 2022] GET /compute/v2.1/servers/2afc317e-fd8d-4e86-89d9-aba4b76d83cb => generated 1266 bytes in 188 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:39:11.760563 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-3e24b3b3-94b8-4f2c-9e1d-d910c2b45ea2 tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Calling method '>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:39:11.761553 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-3e24b3b3-94b8-4f2c-9e1d-d910c2b45ea2 tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] [instance: 2afc317e-fd8d-4e86-89d9-aba4b76d83cb] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:39:11.771972 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-3e24b3b3-94b8-4f2c-9e1d-d910c2b45ea2 tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:39:11.772490 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-3e24b3b3-94b8-4f2c-9e1d-d910c2b45ea2 tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:39:11.772921 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-3e24b3b3-94b8-4f2c-9e1d-d910c2b45ea2 tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:39:11.773683 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-3e24b3b3-94b8-4f2c-9e1d-d910c2b45ea2 tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:39:11.774131 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-3e24b3b3-94b8-4f2c-9e1d-d910c2b45ea2 tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:39:11.774548 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-3e24b3b3-94b8-4f2c-9e1d-d910c2b45ea2 tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:39:11.924689 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-3e24b3b3-94b8-4f2c-9e1d-d910c2b45ea2 tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3fe5a1a037fa460eb3121e4d307542da', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '79e671941c234f36b542080d256de2c6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:39:11.935288 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-3e24b3b3-94b8-4f2c-9e1d-d910c2b45ea2 tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] 10.222.0.52 "GET /compute/v2.1/servers/2afc317e-fd8d-4e86-89d9-aba4b76d83cb" status: 200 len: 1266 microversion: 2.1 time: 0.181140 Aug 27 02:39:11.935991 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 570/1133] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:39:11 2022] GET /compute/v2.1/servers/2afc317e-fd8d-4e86-89d9-aba4b76d83cb => generated 1266 bytes in 183 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:39:12.959966 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-7b204b4c-9f46-4131-97a9-9cb0e9927080 tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:39:12.960585 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-7b204b4c-9f46-4131-97a9-9cb0e9927080 tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] [instance: 2afc317e-fd8d-4e86-89d9-aba4b76d83cb] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:39:12.967643 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-7b204b4c-9f46-4131-97a9-9cb0e9927080 tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:39:12.968052 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-7b204b4c-9f46-4131-97a9-9cb0e9927080 tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:39:12.968492 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-7b204b4c-9f46-4131-97a9-9cb0e9927080 tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:39:12.969226 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-7b204b4c-9f46-4131-97a9-9cb0e9927080 tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:39:12.969690 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-7b204b4c-9f46-4131-97a9-9cb0e9927080 tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:39:12.970124 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-7b204b4c-9f46-4131-97a9-9cb0e9927080 tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:39:13.110491 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-7b204b4c-9f46-4131-97a9-9cb0e9927080 tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3fe5a1a037fa460eb3121e4d307542da', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '79e671941c234f36b542080d256de2c6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:39:13.118821 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-7b204b4c-9f46-4131-97a9-9cb0e9927080 tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] 10.222.0.52 "GET /compute/v2.1/servers/2afc317e-fd8d-4e86-89d9-aba4b76d83cb" status: 200 len: 1454 microversion: 2.1 time: 0.162761 Aug 27 02:39:13.119373 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 564/1134] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:39:12 2022] GET /compute/v2.1/servers/2afc317e-fd8d-4e86-89d9-aba4b76d83cb => generated 1454 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:39:13.139439 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-38a4fb67-9fa6-4b79-bc6d-38163eacdc0e tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Calling method '>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:39:13.140092 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-38a4fb67-9fa6-4b79-bc6d-38163eacdc0e tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] [instance: 2afc317e-fd8d-4e86-89d9-aba4b76d83cb] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:39:13.146629 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-38a4fb67-9fa6-4b79-bc6d-38163eacdc0e tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:39:13.147116 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-38a4fb67-9fa6-4b79-bc6d-38163eacdc0e tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:39:13.147549 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-38a4fb67-9fa6-4b79-bc6d-38163eacdc0e tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:39:13.148242 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-38a4fb67-9fa6-4b79-bc6d-38163eacdc0e tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:39:13.148690 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-38a4fb67-9fa6-4b79-bc6d-38163eacdc0e tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:39:13.149140 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-38a4fb67-9fa6-4b79-bc6d-38163eacdc0e tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:39:13.285899 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-38a4fb67-9fa6-4b79-bc6d-38163eacdc0e tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3fe5a1a037fa460eb3121e4d307542da', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '79e671941c234f36b542080d256de2c6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:39:13.296089 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-38a4fb67-9fa6-4b79-bc6d-38163eacdc0e tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] 10.222.0.52 "GET /compute/v2.1/servers/2afc317e-fd8d-4e86-89d9-aba4b76d83cb" status: 200 len: 1454 microversion: 2.1 time: 0.160506 Aug 27 02:39:13.296673 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 571/1135] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:39:13 2022] GET /compute/v2.1/servers/2afc317e-fd8d-4e86-89d9-aba4b76d83cb => generated 1454 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:39:15.388440 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [req-d9b48cc4-2456-42bc-a96a-e17ac4fa7c75 req-4af90d06-2bb5-4ec1-aaca-60e2fca89fc9 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "2afc317e-fd8d-4e86-89d9-aba4b76d83cb", "tag": "df547cda-45a3-4fd8-bd2f-f3b5e1938759"}]} {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 27 02:39:15.398944 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [req-d9b48cc4-2456-42bc-a96a-e17ac4fa7c75 req-4af90d06-2bb5-4ec1-aaca-60e2fca89fc9 service nova] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:39:15.399793 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [req-d9b48cc4-2456-42bc-a96a-e17ac4fa7c75 req-4af90d06-2bb5-4ec1-aaca-60e2fca89fc9 service nova] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:39:15.400350 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [req-d9b48cc4-2456-42bc-a96a-e17ac4fa7c75 req-4af90d06-2bb5-4ec1-aaca-60e2fca89fc9 service nova] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:39:15.412012 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.compute.server_external_events [req-d9b48cc4-2456-42bc-a96a-e17ac4fa7c75 req-4af90d06-2bb5-4ec1-aaca-60e2fca89fc9 service nova] Creating event network-changed:df547cda-45a3-4fd8-bd2f-f3b5e1938759 for instance 2afc317e-fd8d-4e86-89d9-aba4b76d83cb on hv-cinder-84630 Aug 27 02:39:15.417786 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [req-d9b48cc4-2456-42bc-a96a-e17ac4fa7c75 req-4af90d06-2bb5-4ec1-aaca-60e2fca89fc9 service nova] 10.222.0.52 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.033530 Aug 27 02:39:15.418369 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 565/1136] 10.222.0.52 () {64 vars in 1317 bytes} [Sat Aug 27 02:39:15 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 35 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 27 02:39:54.622073 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-75ac9083-ab9e-4445-90f2-f8b1277ef3ff tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Calling method '>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:39:54.624266 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-75ac9083-ab9e-4445-90f2-f8b1277ef3ff tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] [instance: f5bad70c-350c-4230-a862-626dd6805e4c] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:39:54.626547 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG dbcounter [-] [120864] Writing DB stats nova_api:SELECT=7 {{(pid=120864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 27 02:39:54.627195 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG dbcounter [-] [120864] Writing DB stats nova_cell1:SELECT=32 {{(pid=120864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 27 02:39:54.632688 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-75ac9083-ab9e-4445-90f2-f8b1277ef3ff tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:39:54.633157 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-75ac9083-ab9e-4445-90f2-f8b1277ef3ff tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:39:54.633622 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-75ac9083-ab9e-4445-90f2-f8b1277ef3ff tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:39:54.634319 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-75ac9083-ab9e-4445-90f2-f8b1277ef3ff tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:39:54.634771 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-75ac9083-ab9e-4445-90f2-f8b1277ef3ff tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:39:54.635178 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-75ac9083-ab9e-4445-90f2-f8b1277ef3ff tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:39:54.666242 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-75ac9083-ab9e-4445-90f2-f8b1277ef3ff tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] 10.222.0.52 "GET /compute/v2.1/servers/f5bad70c-350c-4230-a862-626dd6805e4c/os-volume_attachments" status: 200 len: 25 microversion: 2.1 time: 0.048103 Aug 27 02:39:54.666857 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 572/1137] 10.222.0.52 () {58 vars in 1302 bytes} [Sat Aug 27 02:39:54 2022] GET /compute/v2.1/servers/f5bad70c-350c-4230-a862-626dd6805e4c/os-volume_attachments => generated 25 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Aug 27 02:39:54.729444 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG dbcounter [-] [120863] Writing DB stats nova_api:SELECT=9 {{(pid=120863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 27 02:39:54.730236 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG dbcounter [-] [120863] Writing DB stats nova_cell1:SELECT=25 {{(pid=120863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 27 02:39:54.735066 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-50d4eb72-e631-406d-841f-b5f0fd3ef1a6 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:39:54.735659 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-50d4eb72-e631-406d-841f-b5f0fd3ef1a6 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] [instance: f5bad70c-350c-4230-a862-626dd6805e4c] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:39:54.743571 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-50d4eb72-e631-406d-841f-b5f0fd3ef1a6 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:39:54.744106 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-50d4eb72-e631-406d-841f-b5f0fd3ef1a6 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:39:54.744565 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-50d4eb72-e631-406d-841f-b5f0fd3ef1a6 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:39:54.745372 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-50d4eb72-e631-406d-841f-b5f0fd3ef1a6 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:39:54.745780 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-50d4eb72-e631-406d-841f-b5f0fd3ef1a6 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:39:54.746213 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-50d4eb72-e631-406d-841f-b5f0fd3ef1a6 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:39:54.778745 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-50d4eb72-e631-406d-841f-b5f0fd3ef1a6 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] [instance: f5bad70c-350c-4230-a862-626dd6805e4c] Going to try to terminate instance {{(pid=120863) delete /opt/stack/nova/nova/compute/api.py:2719}} Aug 27 02:39:54.866978 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-50d4eb72-e631-406d-841f-b5f0fd3ef1a6 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] 10.222.0.52 "DELETE /compute/v2.1/servers/f5bad70c-350c-4230-a862-626dd6805e4c" status: 204 len: 0 microversion: 2.1 time: 0.139222 Aug 27 02:39:54.867572 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 566/1138] 10.222.0.52 () {58 vars in 1239 bytes} [Sat Aug 27 02:39:54 2022] DELETE /compute/v2.1/servers/f5bad70c-350c-4230-a862-626dd6805e4c => generated 0 bytes in 141 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Aug 27 02:39:54.877399 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-85da9137-adcb-4772-ae87-e7a367a39c47 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Calling method '>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:39:54.878025 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-85da9137-adcb-4772-ae87-e7a367a39c47 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] [instance: f5bad70c-350c-4230-a862-626dd6805e4c] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:39:54.884638 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-85da9137-adcb-4772-ae87-e7a367a39c47 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:39:54.885176 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-85da9137-adcb-4772-ae87-e7a367a39c47 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:39:54.885590 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-85da9137-adcb-4772-ae87-e7a367a39c47 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:39:54.886282 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-85da9137-adcb-4772-ae87-e7a367a39c47 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:39:54.886733 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-85da9137-adcb-4772-ae87-e7a367a39c47 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:39:54.887155 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-85da9137-adcb-4772-ae87-e7a367a39c47 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:39:55.032081 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-85da9137-adcb-4772-ae87-e7a367a39c47 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee6084fffc544dfe8f1a9a30ae06be69', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a8055f4326f640fc8927f5cfdef39dd3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:39:55.045600 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-85da9137-adcb-4772-ae87-e7a367a39c47 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] 10.222.0.52 "GET /compute/v2.1/servers/f5bad70c-350c-4230-a862-626dd6805e4c" status: 200 len: 1672 microversion: 2.1 time: 0.173156 Aug 27 02:39:55.046489 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 573/1139] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:39:54 2022] GET /compute/v2.1/servers/f5bad70c-350c-4230-a862-626dd6805e4c => generated 1672 bytes in 175 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:39:56.072750 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-a8d2108e-c74a-43fb-a115-3d0ee0d978b6 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:39:56.073486 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-a8d2108e-c74a-43fb-a115-3d0ee0d978b6 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] [instance: f5bad70c-350c-4230-a862-626dd6805e4c] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:39:56.083366 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-a8d2108e-c74a-43fb-a115-3d0ee0d978b6 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:39:56.083924 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-a8d2108e-c74a-43fb-a115-3d0ee0d978b6 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:39:56.084366 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-a8d2108e-c74a-43fb-a115-3d0ee0d978b6 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:39:56.085207 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-a8d2108e-c74a-43fb-a115-3d0ee0d978b6 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:39:56.085831 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-a8d2108e-c74a-43fb-a115-3d0ee0d978b6 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:39:56.086434 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-a8d2108e-c74a-43fb-a115-3d0ee0d978b6 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:39:56.245951 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-a8d2108e-c74a-43fb-a115-3d0ee0d978b6 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee6084fffc544dfe8f1a9a30ae06be69', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a8055f4326f640fc8927f5cfdef39dd3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:39:56.257230 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-a8d2108e-c74a-43fb-a115-3d0ee0d978b6 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] 10.222.0.52 "GET /compute/v2.1/servers/f5bad70c-350c-4230-a862-626dd6805e4c" status: 200 len: 1672 microversion: 2.1 time: 0.191526 Aug 27 02:39:56.257863 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 567/1140] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:39:56 2022] GET /compute/v2.1/servers/f5bad70c-350c-4230-a862-626dd6805e4c => generated 1672 bytes in 194 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:39:57.280402 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-7de3d764-a9d1-4eca-88e1-d0570a153863 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Calling method '>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:39:57.281110 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-7de3d764-a9d1-4eca-88e1-d0570a153863 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] [instance: f5bad70c-350c-4230-a862-626dd6805e4c] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:39:57.288966 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-7de3d764-a9d1-4eca-88e1-d0570a153863 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:39:57.289484 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-7de3d764-a9d1-4eca-88e1-d0570a153863 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:39:57.289924 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-7de3d764-a9d1-4eca-88e1-d0570a153863 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:39:57.290671 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-7de3d764-a9d1-4eca-88e1-d0570a153863 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:39:57.291093 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-7de3d764-a9d1-4eca-88e1-d0570a153863 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:39:57.292126 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-7de3d764-a9d1-4eca-88e1-d0570a153863 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:39:57.552813 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-7de3d764-a9d1-4eca-88e1-d0570a153863 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee6084fffc544dfe8f1a9a30ae06be69', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a8055f4326f640fc8927f5cfdef39dd3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:39:57.564293 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-7de3d764-a9d1-4eca-88e1-d0570a153863 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] 10.222.0.52 "GET /compute/v2.1/servers/f5bad70c-350c-4230-a862-626dd6805e4c" status: 200 len: 1672 microversion: 2.1 time: 0.288141 Aug 27 02:39:57.565088 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 574/1141] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:39:57 2022] GET /compute/v2.1/servers/f5bad70c-350c-4230-a862-626dd6805e4c => generated 1672 bytes in 291 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:39:58.163123 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [req-ec6a3f96-7866-4189-a0ad-03c753856753 req-953902a7-3a51-43dc-896d-110c1eba28d1 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "f5bad70c-350c-4230-a862-626dd6805e4c", "name": "network-vif-deleted", "tag": "4e23603f-f7cd-42a0-9751-b3014224edb8"}]} {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 27 02:39:58.176626 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [req-ec6a3f96-7866-4189-a0ad-03c753856753 req-953902a7-3a51-43dc-896d-110c1eba28d1 service nova] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:39:58.177112 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [req-ec6a3f96-7866-4189-a0ad-03c753856753 req-953902a7-3a51-43dc-896d-110c1eba28d1 service nova] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:39:58.177573 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [req-ec6a3f96-7866-4189-a0ad-03c753856753 req-953902a7-3a51-43dc-896d-110c1eba28d1 service nova] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:39:58.190731 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.compute.server_external_events [req-ec6a3f96-7866-4189-a0ad-03c753856753 req-953902a7-3a51-43dc-896d-110c1eba28d1 service nova] Creating event network-vif-deleted:4e23603f-f7cd-42a0-9751-b3014224edb8 for instance f5bad70c-350c-4230-a862-626dd6805e4c on hv-cinder-84630 Aug 27 02:39:58.197931 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [req-ec6a3f96-7866-4189-a0ad-03c753856753 req-953902a7-3a51-43dc-896d-110c1eba28d1 service nova] 10.222.0.52 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.038173 Aug 27 02:39:58.199110 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 568/1142] 10.222.0.52 () {64 vars in 1317 bytes} [Sat Aug 27 02:39:58 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 40 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 27 02:39:58.593573 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-068420ea-fdc9-4278-a150-d9cc6ac2eb08 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Calling method '>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:39:58.594333 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-068420ea-fdc9-4278-a150-d9cc6ac2eb08 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] [instance: f5bad70c-350c-4230-a862-626dd6805e4c] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:39:58.601903 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-068420ea-fdc9-4278-a150-d9cc6ac2eb08 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:39:58.602471 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-068420ea-fdc9-4278-a150-d9cc6ac2eb08 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:39:58.602909 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-068420ea-fdc9-4278-a150-d9cc6ac2eb08 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:39:58.603609 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-068420ea-fdc9-4278-a150-d9cc6ac2eb08 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:39:58.604064 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-068420ea-fdc9-4278-a150-d9cc6ac2eb08 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:39:58.604484 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-068420ea-fdc9-4278-a150-d9cc6ac2eb08 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:39:58.645517 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-068420ea-fdc9-4278-a150-d9cc6ac2eb08 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:39:58.645896 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-068420ea-fdc9-4278-a150-d9cc6ac2eb08 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:39:58.646329 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-068420ea-fdc9-4278-a150-d9cc6ac2eb08 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:39:58.646787 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.objects.instance [None req-068420ea-fdc9-4278-a150-d9cc6ac2eb08 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Lazy-loading 'fault' on Instance uuid f5bad70c-350c-4230-a862-626dd6805e4c {{(pid=120864) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Aug 27 02:39:58.690321 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-068420ea-fdc9-4278-a150-d9cc6ac2eb08 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee6084fffc544dfe8f1a9a30ae06be69', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a8055f4326f640fc8927f5cfdef39dd3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:39:58.700728 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-068420ea-fdc9-4278-a150-d9cc6ac2eb08 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] 10.222.0.52 "GET /compute/v2.1/servers/f5bad70c-350c-4230-a862-626dd6805e4c" status: 200 len: 1327 microversion: 2.1 time: 0.112445 Aug 27 02:39:58.701922 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 575/1143] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:39:58 2022] GET /compute/v2.1/servers/f5bad70c-350c-4230-a862-626dd6805e4c => generated 1327 bytes in 114 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:39:59.725480 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-cb28f34c-3808-4a38-8ad6-5bfc3695bf1d tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:39:59.726189 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-cb28f34c-3808-4a38-8ad6-5bfc3695bf1d tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] [instance: f5bad70c-350c-4230-a862-626dd6805e4c] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:39:59.733145 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-cb28f34c-3808-4a38-8ad6-5bfc3695bf1d tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:39:59.733613 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-cb28f34c-3808-4a38-8ad6-5bfc3695bf1d tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:39:59.734051 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-cb28f34c-3808-4a38-8ad6-5bfc3695bf1d tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:39:59.734756 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-cb28f34c-3808-4a38-8ad6-5bfc3695bf1d tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:39:59.735214 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-cb28f34c-3808-4a38-8ad6-5bfc3695bf1d tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:39:59.735643 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-cb28f34c-3808-4a38-8ad6-5bfc3695bf1d tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:39:59.754831 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.wsgi [None req-cb28f34c-3808-4a38-8ad6-5bfc3695bf1d tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] HTTP exception thrown: Instance f5bad70c-350c-4230-a862-626dd6805e4c could not be found. Aug 27 02:39:59.757258 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-cb28f34c-3808-4a38-8ad6-5bfc3695bf1d tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Request method failure captured: Aug 27 02:39:59.757258 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: request: GET /compute/v2.1/servers/f5bad70c-350c-4230-a862-626dd6805e4c HTTP/1.1 Aug 27 02:39:59.757258 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: Accept: application/json Aug 27 02:39:59.757258 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: Accept-Encoding: identity Aug 27 02:39:59.757258 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: Connection: close Aug 27 02:39:59.757258 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: Content-Length: 0 Aug 27 02:39:59.757258 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: Content-Type: application/json Aug 27 02:39:59.757258 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: Host: 10.222.0.52 Aug 27 02:39:59.757258 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: Openstack-System-Scope: None Aug 27 02:39:59.757258 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: User-Agent: python-urllib3/1.26.11 Aug 27 02:39:59.757258 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-Auth-Token: gAAAAABjCYLxG4xShIaurn6kMgDB_csdcgHMyuwGl_M1BtZ0jwtkVUCxblTUmXzLHTJtK985z6ro9EscRnvVjjQdqUAN11VYU2wySoyxpoA1-56aKSiuDjDA_a8zJHo26N0yzT8tbkGVw0nbFhpKTK-Au7FefDK9t_CH7NJV_TwOXmOddAnhccQ Aug 27 02:39:59.757258 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-Domain-Id: None Aug 27 02:39:59.757258 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-Domain-Name: None Aug 27 02:39:59.757258 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-Identity-Status: Confirmed Aug 27 02:39:59.757258 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-Is-Admin-Project: True Aug 27 02:39:59.757258 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-Project-Domain-Id: default Aug 27 02:39:59.757258 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-Project-Domain-Name: Default Aug 27 02:39:59.757258 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-Project-Id: a8055f4326f640fc8927f5cfdef39dd3 Aug 27 02:39:59.757258 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-Project-Name: tempest-AttachVolumeTestJSON-1495157572 Aug 27 02:39:59.757258 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-Role: reader,member Aug 27 02:39:59.757258 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-Roles: reader,member Aug 27 02:39:59.758953 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-Service-Catalog: [{"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.52:8080", "publicURL": "http://10.222.0.52:8080/v1/AUTH_a8055f4326f640fc8927f5cfdef39dd3"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52:9696/networking"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/compute/v2/a8055f4326f640fc8927f5cfdef39dd3"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/image"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/volume/v3/a8055f4326f640fc8927f5cfdef39dd3"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/identity"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/volume/v3/a8055f4326f640fc8927f5cfdef39dd3"}]}] Aug 27 02:39:59.758953 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-Tenant: tempest-AttachVolumeTestJSON-1495157572 Aug 27 02:39:59.758953 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-Tenant-Id: a8055f4326f640fc8927f5cfdef39dd3 Aug 27 02:39:59.758953 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-Tenant-Name: tempest-AttachVolumeTestJSON-1495157572 Aug 27 02:39:59.758953 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-User: tempest-AttachVolumeTestJSON-1495157572-project Aug 27 02:39:59.758953 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-User-Domain-Id: default Aug 27 02:39:59.758953 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-User-Domain-Name: Default Aug 27 02:39:59.758953 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-User-Id: ee6084fffc544dfe8f1a9a30ae06be69 Aug 27 02:39:59.758953 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-User-Name: tempest-AttachVolumeTestJSON-1495157572-project Aug 27 02:39:59.758953 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: method: > Aug 27 02:39:59.758953 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: exception: Instance f5bad70c-350c-4230-a862-626dd6805e4c could not be found. Aug 27 02:39:59.758953 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: action_args: {'id': 'f5bad70c-350c-4230-a862-626dd6805e4c'} Aug 27 02:39:59.760042 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Aug 27 02:39:59.760042 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 27 02:39:59.760042 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Aug 27 02:39:59.760042 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Aug 27 02:39:59.760042 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Aug 27 02:39:59.760042 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Aug 27 02:39:59.760042 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Aug 27 02:39:59.760042 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Aug 27 02:39:59.760042 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Aug 27 02:39:59.760042 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Aug 27 02:39:59.760042 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance f5bad70c-350c-4230-a862-626dd6805e4c could not be found. Aug 27 02:39:59.760042 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi Aug 27 02:39:59.760042 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 27 02:39:59.760042 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi Aug 27 02:39:59.760042 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 27 02:39:59.760042 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 27 02:39:59.760042 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 27 02:39:59.760042 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Aug 27 02:39:59.760042 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Aug 27 02:39:59.760042 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Aug 27 02:39:59.761480 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Aug 27 02:39:59.761480 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Aug 27 02:39:59.761480 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi instance = self._get_server( Aug 27 02:39:59.761480 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Aug 27 02:39:59.761480 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Aug 27 02:39:59.761480 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Aug 27 02:39:59.761480 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Aug 27 02:39:59.761480 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance f5bad70c-350c-4230-a862-626dd6805e4c could not be found. Aug 27 02:39:59.761480 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi Aug 27 02:39:59.761480 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 27 02:39:59.761480 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi Aug 27 02:39:59.761480 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 27 02:39:59.761480 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 27 02:39:59.761480 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 27 02:39:59.761480 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Aug 27 02:39:59.761480 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Aug 27 02:39:59.761480 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance f5bad70c-350c-4230-a862-626dd6805e4c could not be found. Aug 27 02:39:59.761480 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi Aug 27 02:39:59.762925 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-cb28f34c-3808-4a38-8ad6-5bfc3695bf1d tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Returning 404 to user: Instance f5bad70c-350c-4230-a862-626dd6805e4c could not be found. {{(pid=120863) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Aug 27 02:39:59.762925 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-cb28f34c-3808-4a38-8ad6-5bfc3695bf1d tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] 10.222.0.52 "GET /compute/v2.1/servers/f5bad70c-350c-4230-a862-626dd6805e4c" status: 404 len: 111 microversion: 2.1 time: 0.038359 Aug 27 02:39:59.762925 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 569/1144] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:39:59 2022] GET /compute/v2.1/servers/f5bad70c-350c-4230-a862-626dd6805e4c => generated 111 bytes in 40 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Aug 27 02:39:59.769313 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-8f931755-de1f-44b3-9622-0fd0e989ae50 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Calling method '.version_select at 0x7f6b1e892700>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:39:59.795317 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-8f931755-de1f-44b3-9622-0fd0e989ae50 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] 10.222.0.52 "DELETE /compute/v2.1/os-keypairs/tempest-keypair-1652285511" status: 202 len: 0 microversion: 2.1 time: 0.031389 Aug 27 02:39:59.795906 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 576/1145] 10.222.0.52 () {58 vars in 1221 bytes} [Sat Aug 27 02:39:59 2022] DELETE /compute/v2.1/os-keypairs/tempest-keypair-1652285511 => generated 0 bytes in 33 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Aug 27 02:40:00.493205 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-07e8b6b8-c2ed-4741-931a-3394cdedc52b tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Action: 'create', calling method: .version_select at 0x7f6b1b348280>, body: {"keypair": {"name": "tempest-keypair-630444685", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBP48AaXelWP4UwOKEKd+it94ymL3nbjGhsT0nbxEZKOyhFFYSkI5qCaU7mtdP89SV2fUs2+qUbIpuNfGyBvIR0FYKDlRrUuG6zLcjrse46a4I35CGo8svi2FJiveZaVT0w=="}} {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 27 02:40:00.509301 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.quota [None req-07e8b6b8-c2ed-4741-931a-3394cdedc52b tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Getting quotas for project a8055f4326f640fc8927f5cfdef39dd3. Resources: {'key_pairs'} {{(pid=120863) _get_quotas /opt/stack/nova/nova/quota.py:393}} Aug 27 02:40:00.513510 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.quota [None req-07e8b6b8-c2ed-4741-931a-3394cdedc52b tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Getting quotas for user ee6084fffc544dfe8f1a9a30ae06be69 and project a8055f4326f640fc8927f5cfdef39dd3. Resources: {'key_pairs'} {{(pid=120863) _get_quotas /opt/stack/nova/nova/quota.py:383}} Aug 27 02:40:00.545383 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-07e8b6b8-c2ed-4741-931a-3394cdedc52b tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] 10.222.0.52 "POST /compute/v2.1/os-keypairs" status: 200 len: 385 microversion: 2.1 time: 0.057010 Aug 27 02:40:00.546217 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 570/1146] 10.222.0.52 () {60 vars in 1159 bytes} [Sat Aug 27 02:40:00 2022] POST /compute/v2.1/os-keypairs => generated 385 bytes in 59 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 27 02:40:01.966287 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-c52d6198-c548-415c-9a94-774dc44c4277 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-AttachVolumeTestJSON-server-1421197468", "imageRef": "9fc30b36-b827-46f9-892d-30e808bc2b8a", "flavorRef": "452", "adminPass": "***", "networks": [{"uuid": "59638dff-2144-4bc2-a657-3cc12e54e5c1"}], "security_groups": [{"name": "tempest-securitygroup--2000827975"}], "key_name": "tempest-keypair-630444685", "user_data": "}} {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 27 02:40:02.086547 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.network.neutron [None req-c52d6198-c548-415c-9a94-774dc44c4277 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] validate_networks() for [('59638dff-2144-4bc2-a657-3cc12e54e5c1', None, None, None, None, None)] {{(pid=120864) validate_networks /opt/stack/nova/nova/network/neutron.py:2626}} Aug 27 02:40:02.515436 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.virt.hardware [None req-c52d6198-c548-415c-9a94-774dc44c4277 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Flavor limits 0:0:0 {{(pid=120864) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Aug 27 02:40:02.515866 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.virt.hardware [None req-c52d6198-c548-415c-9a94-774dc44c4277 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Image limits 0:0:0 {{(pid=120864) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Aug 27 02:40:02.516337 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.virt.hardware [None req-c52d6198-c548-415c-9a94-774dc44c4277 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Flavor pref 0:0:0 {{(pid=120864) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Aug 27 02:40:02.516746 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.virt.hardware [None req-c52d6198-c548-415c-9a94-774dc44c4277 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Image pref 0:0:0 {{(pid=120864) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Aug 27 02:40:02.517148 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.virt.hardware [None req-c52d6198-c548-415c-9a94-774dc44c4277 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=120864) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Aug 27 02:40:02.527775 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-c52d6198-c548-415c-9a94-774dc44c4277 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:40:02.528370 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-c52d6198-c548-415c-9a94-774dc44c4277 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:40:02.528711 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-c52d6198-c548-415c-9a94-774dc44c4277 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:40:02.529472 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-c52d6198-c548-415c-9a94-774dc44c4277 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:40:02.529988 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-c52d6198-c548-415c-9a94-774dc44c4277 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:40:02.530351 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-c52d6198-c548-415c-9a94-774dc44c4277 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:40:02.548904 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.quota [None req-c52d6198-c548-415c-9a94-774dc44c4277 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Getting quotas for project a8055f4326f640fc8927f5cfdef39dd3. Resources: {'cores', 'ram', 'instances'} {{(pid=120864) _get_quotas /opt/stack/nova/nova/quota.py:393}} Aug 27 02:40:02.553256 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.quota [None req-c52d6198-c548-415c-9a94-774dc44c4277 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Getting quotas for user ee6084fffc544dfe8f1a9a30ae06be69 and project a8055f4326f640fc8927f5cfdef39dd3. Resources: {'cores', 'ram', 'instances'} {{(pid=120864) _get_quotas /opt/stack/nova/nova/quota.py:383}} Aug 27 02:40:02.565142 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-c52d6198-c548-415c-9a94-774dc44c4277 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Going to run 1 instances... {{(pid=120864) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Aug 27 02:40:02.567770 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-c52d6198-c548-415c-9a94-774dc44c4277 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] [instance: 7f0ec822-1c41-4508-8d13-a48c4c88f133] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='9fc30b36-b827-46f9-892d-30e808bc2b8a',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=120864) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Aug 27 02:40:02.621904 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-c52d6198-c548-415c-9a94-774dc44c4277 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] 10.222.0.52 "POST /compute/v2.1/servers" status: 202 len: 402 microversion: 2.1 time: 0.660391 Aug 27 02:40:02.622559 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 577/1147] 10.222.0.52 () {60 vars in 1147 bytes} [Sat Aug 27 02:40:01 2022] POST /compute/v2.1/servers => generated 402 bytes in 662 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Aug 27 02:40:02.639970 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-83b21e51-44f1-43bf-960f-be1f7b4ab92d tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:40:02.640958 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-83b21e51-44f1-43bf-960f-be1f7b4ab92d tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] [instance: 7f0ec822-1c41-4508-8d13-a48c4c88f133] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:40:02.697637 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-83b21e51-44f1-43bf-960f-be1f7b4ab92d tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee6084fffc544dfe8f1a9a30ae06be69', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a8055f4326f640fc8927f5cfdef39dd3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:40:02.704714 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-83b21e51-44f1-43bf-960f-be1f7b4ab92d tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] 10.222.0.52 "GET /compute/v2.1/servers/7f0ec822-1c41-4508-8d13-a48c4c88f133" status: 200 len: 1237 microversion: 2.1 time: 0.071603 Aug 27 02:40:02.705303 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 571/1148] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:40:02 2022] GET /compute/v2.1/servers/7f0ec822-1c41-4508-8d13-a48c4c88f133 => generated 1237 bytes in 73 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:40:03.726768 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-03498b39-d217-4821-8626-f0268afaf0b6 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Calling method '>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:40:03.727352 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-03498b39-d217-4821-8626-f0268afaf0b6 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] [instance: 7f0ec822-1c41-4508-8d13-a48c4c88f133] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:40:03.734012 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-03498b39-d217-4821-8626-f0268afaf0b6 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:40:03.734507 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-03498b39-d217-4821-8626-f0268afaf0b6 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:40:03.734934 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-03498b39-d217-4821-8626-f0268afaf0b6 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:40:03.735629 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-03498b39-d217-4821-8626-f0268afaf0b6 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:40:03.736082 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-03498b39-d217-4821-8626-f0268afaf0b6 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:40:03.736503 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-03498b39-d217-4821-8626-f0268afaf0b6 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:40:03.821389 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-03498b39-d217-4821-8626-f0268afaf0b6 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee6084fffc544dfe8f1a9a30ae06be69', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a8055f4326f640fc8927f5cfdef39dd3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:40:03.828793 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-03498b39-d217-4821-8626-f0268afaf0b6 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] 10.222.0.52 "GET /compute/v2.1/servers/7f0ec822-1c41-4508-8d13-a48c4c88f133" status: 200 len: 1297 microversion: 2.1 time: 0.106430 Aug 27 02:40:03.829735 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 578/1149] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:40:03 2022] GET /compute/v2.1/servers/7f0ec822-1c41-4508-8d13-a48c4c88f133 => generated 1297 bytes in 108 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:40:04.850665 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-6133de8a-4715-4cb3-91e5-42b47e427d62 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:40:04.851337 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-6133de8a-4715-4cb3-91e5-42b47e427d62 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] [instance: 7f0ec822-1c41-4508-8d13-a48c4c88f133] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:40:04.858433 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-6133de8a-4715-4cb3-91e5-42b47e427d62 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:40:04.858928 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-6133de8a-4715-4cb3-91e5-42b47e427d62 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:40:04.859369 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-6133de8a-4715-4cb3-91e5-42b47e427d62 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:40:04.860077 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-6133de8a-4715-4cb3-91e5-42b47e427d62 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:40:04.860536 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-6133de8a-4715-4cb3-91e5-42b47e427d62 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:40:04.860967 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-6133de8a-4715-4cb3-91e5-42b47e427d62 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:40:05.346655 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-6133de8a-4715-4cb3-91e5-42b47e427d62 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee6084fffc544dfe8f1a9a30ae06be69', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a8055f4326f640fc8927f5cfdef39dd3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:40:05.355388 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-6133de8a-4715-4cb3-91e5-42b47e427d62 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] 10.222.0.52 "GET /compute/v2.1/servers/7f0ec822-1c41-4508-8d13-a48c4c88f133" status: 200 len: 1363 microversion: 2.1 time: 0.508311 Aug 27 02:40:05.355994 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 572/1150] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:40:04 2022] GET /compute/v2.1/servers/7f0ec822-1c41-4508-8d13-a48c4c88f133 => generated 1363 bytes in 510 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:40:05.642207 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [req-12e41c71-89c9-4a40-94a0-eb7111959c05 req-58922ca3-a528-458e-80e4-8e9aa12ed23e service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "7f0ec822-1c41-4508-8d13-a48c4c88f133", "tag": "5d767859-f041-4829-989b-8f1b21644308"}]} {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 27 02:40:05.654394 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [req-12e41c71-89c9-4a40-94a0-eb7111959c05 req-58922ca3-a528-458e-80e4-8e9aa12ed23e service nova] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:40:05.655066 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [req-12e41c71-89c9-4a40-94a0-eb7111959c05 req-58922ca3-a528-458e-80e4-8e9aa12ed23e service nova] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:40:05.656191 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [req-12e41c71-89c9-4a40-94a0-eb7111959c05 req-58922ca3-a528-458e-80e4-8e9aa12ed23e service nova] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:40:05.668580 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.compute.server_external_events [req-12e41c71-89c9-4a40-94a0-eb7111959c05 req-58922ca3-a528-458e-80e4-8e9aa12ed23e service nova] Creating event network-changed:5d767859-f041-4829-989b-8f1b21644308 for instance 7f0ec822-1c41-4508-8d13-a48c4c88f133 on hv-cinder-84630 Aug 27 02:40:05.674126 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [req-12e41c71-89c9-4a40-94a0-eb7111959c05 req-58922ca3-a528-458e-80e4-8e9aa12ed23e service nova] 10.222.0.52 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.035415 Aug 27 02:40:05.674982 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 579/1151] 10.222.0.52 () {64 vars in 1317 bytes} [Sat Aug 27 02:40:05 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 37 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 27 02:40:06.376406 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-68015d51-d474-4a63-a97e-ded57c24d0c0 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:40:06.377031 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-68015d51-d474-4a63-a97e-ded57c24d0c0 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] [instance: 7f0ec822-1c41-4508-8d13-a48c4c88f133] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:40:06.384041 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-68015d51-d474-4a63-a97e-ded57c24d0c0 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:40:06.384613 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-68015d51-d474-4a63-a97e-ded57c24d0c0 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:40:06.385100 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-68015d51-d474-4a63-a97e-ded57c24d0c0 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:40:06.385898 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-68015d51-d474-4a63-a97e-ded57c24d0c0 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:40:06.386345 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-68015d51-d474-4a63-a97e-ded57c24d0c0 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:40:06.386972 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-68015d51-d474-4a63-a97e-ded57c24d0c0 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:40:06.537688 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-68015d51-d474-4a63-a97e-ded57c24d0c0 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee6084fffc544dfe8f1a9a30ae06be69', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a8055f4326f640fc8927f5cfdef39dd3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:40:06.546068 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-68015d51-d474-4a63-a97e-ded57c24d0c0 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] 10.222.0.52 "GET /compute/v2.1/servers/7f0ec822-1c41-4508-8d13-a48c4c88f133" status: 200 len: 1363 microversion: 2.1 time: 0.173507 Aug 27 02:40:06.546637 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 573/1152] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:40:06 2022] GET /compute/v2.1/servers/7f0ec822-1c41-4508-8d13-a48c4c88f133 => generated 1363 bytes in 175 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:40:07.570169 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-f962c586-c549-4779-99fb-a41ae5f18217 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Calling method '>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:40:07.571204 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-f962c586-c549-4779-99fb-a41ae5f18217 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] [instance: 7f0ec822-1c41-4508-8d13-a48c4c88f133] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:40:07.578367 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-f962c586-c549-4779-99fb-a41ae5f18217 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:40:07.578846 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-f962c586-c549-4779-99fb-a41ae5f18217 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:40:07.579273 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-f962c586-c549-4779-99fb-a41ae5f18217 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:40:07.579977 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-f962c586-c549-4779-99fb-a41ae5f18217 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:40:07.580799 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-f962c586-c549-4779-99fb-a41ae5f18217 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:40:07.581384 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-f962c586-c549-4779-99fb-a41ae5f18217 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:40:07.724275 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-f962c586-c549-4779-99fb-a41ae5f18217 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee6084fffc544dfe8f1a9a30ae06be69', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a8055f4326f640fc8927f5cfdef39dd3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:40:07.735318 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-f962c586-c549-4779-99fb-a41ae5f18217 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] 10.222.0.52 "GET /compute/v2.1/servers/7f0ec822-1c41-4508-8d13-a48c4c88f133" status: 200 len: 1363 microversion: 2.1 time: 0.170068 Aug 27 02:40:07.736102 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 580/1153] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:40:07 2022] GET /compute/v2.1/servers/7f0ec822-1c41-4508-8d13-a48c4c88f133 => generated 1363 bytes in 172 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:40:08.757216 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-36033818-5510-44b7-9379-56e22dd3b627 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:40:08.757907 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-36033818-5510-44b7-9379-56e22dd3b627 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] [instance: 7f0ec822-1c41-4508-8d13-a48c4c88f133] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:40:08.765036 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-36033818-5510-44b7-9379-56e22dd3b627 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:40:08.765533 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-36033818-5510-44b7-9379-56e22dd3b627 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:40:08.765992 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-36033818-5510-44b7-9379-56e22dd3b627 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:40:08.766808 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-36033818-5510-44b7-9379-56e22dd3b627 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:40:08.767152 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-36033818-5510-44b7-9379-56e22dd3b627 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:40:08.767833 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-36033818-5510-44b7-9379-56e22dd3b627 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:40:09.001426 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-36033818-5510-44b7-9379-56e22dd3b627 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee6084fffc544dfe8f1a9a30ae06be69', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a8055f4326f640fc8927f5cfdef39dd3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:40:09.010388 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-36033818-5510-44b7-9379-56e22dd3b627 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] 10.222.0.52 "GET /compute/v2.1/servers/7f0ec822-1c41-4508-8d13-a48c4c88f133" status: 200 len: 1363 microversion: 2.1 time: 0.257008 Aug 27 02:40:09.011016 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 574/1154] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:40:08 2022] GET /compute/v2.1/servers/7f0ec822-1c41-4508-8d13-a48c4c88f133 => generated 1363 bytes in 258 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:40:10.031619 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-29016c51-c211-4781-85dd-d06959931541 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Calling method '>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:40:10.033017 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-29016c51-c211-4781-85dd-d06959931541 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] [instance: 7f0ec822-1c41-4508-8d13-a48c4c88f133] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:40:10.044416 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-29016c51-c211-4781-85dd-d06959931541 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:40:10.044911 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-29016c51-c211-4781-85dd-d06959931541 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:40:10.045369 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-29016c51-c211-4781-85dd-d06959931541 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:40:10.046154 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-29016c51-c211-4781-85dd-d06959931541 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:40:10.046697 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-29016c51-c211-4781-85dd-d06959931541 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:40:10.047406 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-29016c51-c211-4781-85dd-d06959931541 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:40:10.193031 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-29016c51-c211-4781-85dd-d06959931541 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee6084fffc544dfe8f1a9a30ae06be69', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a8055f4326f640fc8927f5cfdef39dd3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:40:10.201385 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-29016c51-c211-4781-85dd-d06959931541 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] 10.222.0.52 "GET /compute/v2.1/servers/7f0ec822-1c41-4508-8d13-a48c4c88f133" status: 200 len: 1363 microversion: 2.1 time: 0.173713 Aug 27 02:40:10.201961 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 581/1155] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:40:10 2022] GET /compute/v2.1/servers/7f0ec822-1c41-4508-8d13-a48c4c88f133 => generated 1363 bytes in 175 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:40:11.223599 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-3f870baa-a636-4d47-9417-cbee8e8b5789 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:40:11.224161 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-3f870baa-a636-4d47-9417-cbee8e8b5789 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] [instance: 7f0ec822-1c41-4508-8d13-a48c4c88f133] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:40:11.230889 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-3f870baa-a636-4d47-9417-cbee8e8b5789 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:40:11.231368 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-3f870baa-a636-4d47-9417-cbee8e8b5789 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:40:11.231801 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-3f870baa-a636-4d47-9417-cbee8e8b5789 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:40:11.232495 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-3f870baa-a636-4d47-9417-cbee8e8b5789 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:40:11.232941 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-3f870baa-a636-4d47-9417-cbee8e8b5789 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:40:11.233393 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-3f870baa-a636-4d47-9417-cbee8e8b5789 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:40:11.384485 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-3f870baa-a636-4d47-9417-cbee8e8b5789 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee6084fffc544dfe8f1a9a30ae06be69', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a8055f4326f640fc8927f5cfdef39dd3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:40:11.394426 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-3f870baa-a636-4d47-9417-cbee8e8b5789 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] 10.222.0.52 "GET /compute/v2.1/servers/7f0ec822-1c41-4508-8d13-a48c4c88f133" status: 200 len: 1363 microversion: 2.1 time: 0.175257 Aug 27 02:40:11.395003 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 575/1156] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:40:11 2022] GET /compute/v2.1/servers/7f0ec822-1c41-4508-8d13-a48c4c88f133 => generated 1363 bytes in 177 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:40:12.172545 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [req-60831608-b835-4f27-a87c-857873a31867 req-f7411433-be69-4de7-9cf0-f1840bae2745 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "7f0ec822-1c41-4508-8d13-a48c4c88f133", "name": "network-vif-plugged", "status": "completed", "tag": "5d767859-f041-4829-989b-8f1b21644308"}]} {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 27 02:40:12.183665 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [req-60831608-b835-4f27-a87c-857873a31867 req-f7411433-be69-4de7-9cf0-f1840bae2745 service nova] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:40:12.183665 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [req-60831608-b835-4f27-a87c-857873a31867 req-f7411433-be69-4de7-9cf0-f1840bae2745 service nova] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:40:12.183993 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [req-60831608-b835-4f27-a87c-857873a31867 req-f7411433-be69-4de7-9cf0-f1840bae2745 service nova] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:40:12.196986 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.compute.server_external_events [req-60831608-b835-4f27-a87c-857873a31867 req-f7411433-be69-4de7-9cf0-f1840bae2745 service nova] Creating event network-vif-plugged:5d767859-f041-4829-989b-8f1b21644308 for instance 7f0ec822-1c41-4508-8d13-a48c4c88f133 on hv-cinder-84630 Aug 27 02:40:12.202128 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [req-60831608-b835-4f27-a87c-857873a31867 req-f7411433-be69-4de7-9cf0-f1840bae2745 service nova] 10.222.0.52 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.033376 Aug 27 02:40:12.203084 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 582/1157] 10.222.0.52 () {64 vars in 1317 bytes} [Sat Aug 27 02:40:12 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 35 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 27 02:40:12.418084 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-303e061a-b3c0-49b8-8e76-4fd461488028 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:40:12.418883 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-303e061a-b3c0-49b8-8e76-4fd461488028 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] [instance: 7f0ec822-1c41-4508-8d13-a48c4c88f133] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:40:12.426862 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-303e061a-b3c0-49b8-8e76-4fd461488028 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:40:12.427715 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-303e061a-b3c0-49b8-8e76-4fd461488028 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:40:12.427825 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-303e061a-b3c0-49b8-8e76-4fd461488028 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:40:12.428476 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-303e061a-b3c0-49b8-8e76-4fd461488028 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:40:12.428937 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-303e061a-b3c0-49b8-8e76-4fd461488028 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:40:12.429400 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-303e061a-b3c0-49b8-8e76-4fd461488028 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:40:12.573479 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-303e061a-b3c0-49b8-8e76-4fd461488028 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee6084fffc544dfe8f1a9a30ae06be69', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a8055f4326f640fc8927f5cfdef39dd3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:40:12.582337 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-303e061a-b3c0-49b8-8e76-4fd461488028 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] 10.222.0.52 "GET /compute/v2.1/servers/7f0ec822-1c41-4508-8d13-a48c4c88f133" status: 200 len: 1363 microversion: 2.1 time: 0.168102 Aug 27 02:40:12.582944 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 576/1158] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:40:12 2022] GET /compute/v2.1/servers/7f0ec822-1c41-4508-8d13-a48c4c88f133 => generated 1363 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:40:13.601579 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG dbcounter [-] [120864] Writing DB stats nova_cell0:SELECT=2 {{(pid=120864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 27 02:40:13.602351 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG dbcounter [-] [120864] Writing DB stats nova_cell0:SELECT=6 {{(pid=120864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 27 02:40:13.607274 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-674f4998-844d-4f29-94a7-ce41788d0b13 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Calling method '>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:40:13.607969 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-674f4998-844d-4f29-94a7-ce41788d0b13 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] [instance: 7f0ec822-1c41-4508-8d13-a48c4c88f133] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:40:13.615774 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-674f4998-844d-4f29-94a7-ce41788d0b13 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:40:13.616285 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-674f4998-844d-4f29-94a7-ce41788d0b13 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:40:13.616715 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-674f4998-844d-4f29-94a7-ce41788d0b13 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:40:13.617431 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-674f4998-844d-4f29-94a7-ce41788d0b13 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:40:13.617881 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-674f4998-844d-4f29-94a7-ce41788d0b13 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:40:13.618299 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-674f4998-844d-4f29-94a7-ce41788d0b13 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:40:13.760925 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-674f4998-844d-4f29-94a7-ce41788d0b13 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee6084fffc544dfe8f1a9a30ae06be69', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a8055f4326f640fc8927f5cfdef39dd3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:40:13.769367 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-674f4998-844d-4f29-94a7-ce41788d0b13 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] 10.222.0.52 "GET /compute/v2.1/servers/7f0ec822-1c41-4508-8d13-a48c4c88f133" status: 200 len: 1363 microversion: 2.1 time: 0.168832 Aug 27 02:40:13.769976 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 583/1159] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:40:13 2022] GET /compute/v2.1/servers/7f0ec822-1c41-4508-8d13-a48c4c88f133 => generated 1363 bytes in 170 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:40:14.790963 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-26f5499a-cb19-46ed-bcf1-aa3ff30ee637 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:40:14.791605 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-26f5499a-cb19-46ed-bcf1-aa3ff30ee637 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] [instance: 7f0ec822-1c41-4508-8d13-a48c4c88f133] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:40:14.793513 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG dbcounter [-] [120863] Writing DB stats nova_cell0:SELECT=6 {{(pid=120863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 27 02:40:14.799841 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-26f5499a-cb19-46ed-bcf1-aa3ff30ee637 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:40:14.800312 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-26f5499a-cb19-46ed-bcf1-aa3ff30ee637 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:40:14.800949 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-26f5499a-cb19-46ed-bcf1-aa3ff30ee637 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:40:14.801657 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-26f5499a-cb19-46ed-bcf1-aa3ff30ee637 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:40:14.802108 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-26f5499a-cb19-46ed-bcf1-aa3ff30ee637 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:40:14.802526 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-26f5499a-cb19-46ed-bcf1-aa3ff30ee637 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:40:14.941014 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-26f5499a-cb19-46ed-bcf1-aa3ff30ee637 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee6084fffc544dfe8f1a9a30ae06be69', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a8055f4326f640fc8927f5cfdef39dd3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:40:14.950189 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-26f5499a-cb19-46ed-bcf1-aa3ff30ee637 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] 10.222.0.52 "GET /compute/v2.1/servers/7f0ec822-1c41-4508-8d13-a48c4c88f133" status: 200 len: 1548 microversion: 2.1 time: 0.163146 Aug 27 02:40:14.950754 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 577/1160] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:40:14 2022] GET /compute/v2.1/servers/7f0ec822-1c41-4508-8d13-a48c4c88f133 => generated 1548 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:40:14.971783 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-c6dd3dbd-1f47-4480-b0aa-e9c386082c9d tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Calling method '>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:40:14.972411 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-c6dd3dbd-1f47-4480-b0aa-e9c386082c9d tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] [instance: 7f0ec822-1c41-4508-8d13-a48c4c88f133] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:40:14.979322 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-c6dd3dbd-1f47-4480-b0aa-e9c386082c9d tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:40:14.979894 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-c6dd3dbd-1f47-4480-b0aa-e9c386082c9d tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:40:14.980324 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-c6dd3dbd-1f47-4480-b0aa-e9c386082c9d tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:40:14.981027 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-c6dd3dbd-1f47-4480-b0aa-e9c386082c9d tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:40:14.981496 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-c6dd3dbd-1f47-4480-b0aa-e9c386082c9d tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:40:14.981916 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-c6dd3dbd-1f47-4480-b0aa-e9c386082c9d tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:40:15.062475 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-c6dd3dbd-1f47-4480-b0aa-e9c386082c9d tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] 10.222.0.52 "GET /compute/v2.1/servers/7f0ec822-1c41-4508-8d13-a48c4c88f133/os-interface" status: 200 len: 284 microversion: 2.1 time: 0.094786 Aug 27 02:40:15.063067 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 584/1161] 10.222.0.52 () {58 vars in 1275 bytes} [Sat Aug 27 02:40:14 2022] GET /compute/v2.1/servers/7f0ec822-1c41-4508-8d13-a48c4c88f133/os-interface => generated 284 bytes in 96 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 27 02:40:16.050380 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [req-fd231b7f-b7be-4e4a-a817-9ba4e81fdbb2 req-7492f7b2-5c72-4f50-ba59-fc0ce51312a2 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "7f0ec822-1c41-4508-8d13-a48c4c88f133", "tag": "5d767859-f041-4829-989b-8f1b21644308"}]} {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 27 02:40:16.060747 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [req-fd231b7f-b7be-4e4a-a817-9ba4e81fdbb2 req-7492f7b2-5c72-4f50-ba59-fc0ce51312a2 service nova] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:40:16.061243 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [req-fd231b7f-b7be-4e4a-a817-9ba4e81fdbb2 req-7492f7b2-5c72-4f50-ba59-fc0ce51312a2 service nova] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:40:16.061669 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [req-fd231b7f-b7be-4e4a-a817-9ba4e81fdbb2 req-7492f7b2-5c72-4f50-ba59-fc0ce51312a2 service nova] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:40:16.073475 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.compute.server_external_events [req-fd231b7f-b7be-4e4a-a817-9ba4e81fdbb2 req-7492f7b2-5c72-4f50-ba59-fc0ce51312a2 service nova] Creating event network-changed:5d767859-f041-4829-989b-8f1b21644308 for instance 7f0ec822-1c41-4508-8d13-a48c4c88f133 on hv-cinder-84630 Aug 27 02:40:16.078740 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [req-fd231b7f-b7be-4e4a-a817-9ba4e81fdbb2 req-7492f7b2-5c72-4f50-ba59-fc0ce51312a2 service nova] 10.222.0.52 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.033796 Aug 27 02:40:16.080075 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 578/1162] 10.222.0.52 () {64 vars in 1317 bytes} [Sat Aug 27 02:40:16 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 36 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 27 02:41:26.586199 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG dbcounter [-] [120864] Writing DB stats nova_api:SELECT=42,nova_api:DELETE=1,nova_api:INSERT=3 {{(pid=120864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 27 02:41:26.586979 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG dbcounter [-] [120864] Writing DB stats nova_cell1:SELECT=60 {{(pid=120864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 27 02:41:26.592818 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-572e4f03-bf44-4111-a533-df6fa9eebcd9 tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-TestVolumeBootPattern-server-1576013670", "imageRef": "", "flavorRef": "452", "key_name": "tempest-TestVolumeBootPattern-1366561407", "security_groups": [{"name": "tempest-secgroup-smoke-743539963"}], "block_device_mapping_v2": [{"uuid": "2d4ff40a-0e60-4c13-bf81-3d7ab6828321", "source_type": "volume", "destination_type": "volume", "boot_index": 0, "delete_on_termination": false}], "networks": [{"uuid": "3a1ac093-e4b6-48ea-aff8-2575f231ba35"}]}} {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 27 02:41:26.730747 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.network.neutron [None req-572e4f03-bf44-4111-a533-df6fa9eebcd9 tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] validate_networks() for [('3a1ac093-e4b6-48ea-aff8-2575f231ba35', None, None, None, None, None)] {{(pid=120864) validate_networks /opt/stack/nova/nova/network/neutron.py:2626}} Aug 27 02:41:27.197491 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.virt.hardware [None req-572e4f03-bf44-4111-a533-df6fa9eebcd9 tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Flavor limits 0:0:0 {{(pid=120864) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Aug 27 02:41:27.198064 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.virt.hardware [None req-572e4f03-bf44-4111-a533-df6fa9eebcd9 tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Image limits 0:0:0 {{(pid=120864) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Aug 27 02:41:27.198528 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.virt.hardware [None req-572e4f03-bf44-4111-a533-df6fa9eebcd9 tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Flavor pref 0:0:0 {{(pid=120864) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Aug 27 02:41:27.198929 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.virt.hardware [None req-572e4f03-bf44-4111-a533-df6fa9eebcd9 tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Image pref 0:0:0 {{(pid=120864) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Aug 27 02:41:27.199307 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.virt.hardware [None req-572e4f03-bf44-4111-a533-df6fa9eebcd9 tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=120864) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Aug 27 02:41:27.209096 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-572e4f03-bf44-4111-a533-df6fa9eebcd9 tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:41:27.209597 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-572e4f03-bf44-4111-a533-df6fa9eebcd9 tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:41:27.210022 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-572e4f03-bf44-4111-a533-df6fa9eebcd9 tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:41:27.210735 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-572e4f03-bf44-4111-a533-df6fa9eebcd9 tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:41:27.211180 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-572e4f03-bf44-4111-a533-df6fa9eebcd9 tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:41:27.211597 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-572e4f03-bf44-4111-a533-df6fa9eebcd9 tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:41:27.229631 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.quota [None req-572e4f03-bf44-4111-a533-df6fa9eebcd9 tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Getting quotas for project 79e671941c234f36b542080d256de2c6. Resources: {'cores', 'ram', 'instances'} {{(pid=120864) _get_quotas /opt/stack/nova/nova/quota.py:393}} Aug 27 02:41:27.233782 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.quota [None req-572e4f03-bf44-4111-a533-df6fa9eebcd9 tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Getting quotas for user 3fe5a1a037fa460eb3121e4d307542da and project 79e671941c234f36b542080d256de2c6. Resources: {'cores', 'ram', 'instances'} {{(pid=120864) _get_quotas /opt/stack/nova/nova/quota.py:383}} Aug 27 02:41:27.298550 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-572e4f03-bf44-4111-a533-df6fa9eebcd9 tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Going to run 1 instances... {{(pid=120864) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Aug 27 02:41:27.300665 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-572e4f03-bf44-4111-a533-df6fa9eebcd9 tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] [instance: a29b4020-7f17-480c-b0f4-062c82eaddd2] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=False,deleted=,deleted_at=,destination_type='volume',device_name=None,device_type=None,disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id=None,instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='volume',tag=None,updated_at=,uuid=,volume_id='2d4ff40a-0e60-4c13-bf81-3d7ab6828321',volume_size=None,volume_type=None)] {{(pid=120864) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Aug 27 02:41:27.454419 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-572e4f03-bf44-4111-a533-df6fa9eebcd9 tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] 10.222.0.52 "POST /compute/v2.1/servers" status: 202 len: 405 microversion: 2.1 time: 0.870421 Aug 27 02:41:27.454975 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 585/1163] 10.222.0.52 () {60 vars in 1147 bytes} [Sat Aug 27 02:41:26 2022] POST /compute/v2.1/servers => generated 405 bytes in 872 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Aug 27 02:41:27.469168 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG dbcounter [-] [120863] Writing DB stats nova_api:SELECT=42,nova_api:UPDATE=1,nova_api:INSERT=1 {{(pid=120863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 27 02:41:27.469994 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG dbcounter [-] [120863] Writing DB stats nova_cell1:SELECT=62,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=120863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 27 02:41:27.477609 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-2d458536-8bf3-4bc4-b39b-ba3901306fe6 tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:41:27.478307 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-2d458536-8bf3-4bc4-b39b-ba3901306fe6 tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] [instance: a29b4020-7f17-480c-b0f4-062c82eaddd2] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:41:27.537612 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-2d458536-8bf3-4bc4-b39b-ba3901306fe6 tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3fe5a1a037fa460eb3121e4d307542da', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '79e671941c234f36b542080d256de2c6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:41:27.544669 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-2d458536-8bf3-4bc4-b39b-ba3901306fe6 tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] 10.222.0.52 "GET /compute/v2.1/servers/a29b4020-7f17-480c-b0f4-062c82eaddd2" status: 200 len: 1095 microversion: 2.1 time: 0.078607 Aug 27 02:41:27.545263 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 579/1164] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:41:27 2022] GET /compute/v2.1/servers/a29b4020-7f17-480c-b0f4-062c82eaddd2 => generated 1095 bytes in 80 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:41:28.569262 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-529f2ae5-636e-4579-99de-3252bc06d48f tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Calling method '>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:41:28.569877 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-529f2ae5-636e-4579-99de-3252bc06d48f tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] [instance: a29b4020-7f17-480c-b0f4-062c82eaddd2] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:41:28.578318 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-529f2ae5-636e-4579-99de-3252bc06d48f tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:41:28.578821 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-529f2ae5-636e-4579-99de-3252bc06d48f tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:41:28.579253 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-529f2ae5-636e-4579-99de-3252bc06d48f tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:41:28.580035 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-529f2ae5-636e-4579-99de-3252bc06d48f tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:41:28.580498 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-529f2ae5-636e-4579-99de-3252bc06d48f tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:41:28.580923 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-529f2ae5-636e-4579-99de-3252bc06d48f tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:41:28.643872 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-529f2ae5-636e-4579-99de-3252bc06d48f tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3fe5a1a037fa460eb3121e4d307542da', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '79e671941c234f36b542080d256de2c6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:41:28.652271 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-529f2ae5-636e-4579-99de-3252bc06d48f tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] 10.222.0.52 "GET /compute/v2.1/servers/a29b4020-7f17-480c-b0f4-062c82eaddd2" status: 200 len: 1193 microversion: 2.1 time: 0.087430 Aug 27 02:41:28.652816 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 586/1165] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:41:28 2022] GET /compute/v2.1/servers/a29b4020-7f17-480c-b0f4-062c82eaddd2 => generated 1193 bytes in 89 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:41:29.680458 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-ff2eee5b-773f-43bd-9d22-56657a255f8a tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:41:29.682138 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-ff2eee5b-773f-43bd-9d22-56657a255f8a tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] [instance: a29b4020-7f17-480c-b0f4-062c82eaddd2] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:41:29.690268 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-ff2eee5b-773f-43bd-9d22-56657a255f8a tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:41:29.691044 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-ff2eee5b-773f-43bd-9d22-56657a255f8a tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:41:29.691473 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-ff2eee5b-773f-43bd-9d22-56657a255f8a tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:41:29.692166 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-ff2eee5b-773f-43bd-9d22-56657a255f8a tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:41:29.695094 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-ff2eee5b-773f-43bd-9d22-56657a255f8a tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:41:29.695737 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-ff2eee5b-773f-43bd-9d22-56657a255f8a tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:41:29.957877 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-ff2eee5b-773f-43bd-9d22-56657a255f8a tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3fe5a1a037fa460eb3121e4d307542da', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '79e671941c234f36b542080d256de2c6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:41:29.966179 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-ff2eee5b-773f-43bd-9d22-56657a255f8a tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] 10.222.0.52 "GET /compute/v2.1/servers/a29b4020-7f17-480c-b0f4-062c82eaddd2" status: 200 len: 1266 microversion: 2.1 time: 0.293597 Aug 27 02:41:29.966787 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 580/1166] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:41:29 2022] GET /compute/v2.1/servers/a29b4020-7f17-480c-b0f4-062c82eaddd2 => generated 1266 bytes in 295 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:41:30.379936 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [req-8984d905-2490-464d-9c5b-455b7a639bfa req-0d1d65e4-ec81-4f03-aefe-097b4317f504 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "a29b4020-7f17-480c-b0f4-062c82eaddd2", "tag": "0f6ab26e-c941-4e20-b27e-ed8630548227"}]} {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 27 02:41:30.390529 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [req-8984d905-2490-464d-9c5b-455b7a639bfa req-0d1d65e4-ec81-4f03-aefe-097b4317f504 service nova] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:41:30.391060 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [req-8984d905-2490-464d-9c5b-455b7a639bfa req-0d1d65e4-ec81-4f03-aefe-097b4317f504 service nova] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:41:30.391485 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [req-8984d905-2490-464d-9c5b-455b7a639bfa req-0d1d65e4-ec81-4f03-aefe-097b4317f504 service nova] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:41:30.403354 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.compute.server_external_events [req-8984d905-2490-464d-9c5b-455b7a639bfa req-0d1d65e4-ec81-4f03-aefe-097b4317f504 service nova] Creating event network-changed:0f6ab26e-c941-4e20-b27e-ed8630548227 for instance a29b4020-7f17-480c-b0f4-062c82eaddd2 on hv-cinder-84630 Aug 27 02:41:30.408526 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [req-8984d905-2490-464d-9c5b-455b7a639bfa req-0d1d65e4-ec81-4f03-aefe-097b4317f504 service nova] 10.222.0.52 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.103982 Aug 27 02:41:30.409143 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 587/1167] 10.222.0.52 () {64 vars in 1317 bytes} [Sat Aug 27 02:41:30 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 105 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 27 02:41:30.987935 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-12a07d4b-0a00-4981-9b30-6a4b6f854bcc tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:41:30.988530 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-12a07d4b-0a00-4981-9b30-6a4b6f854bcc tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] [instance: a29b4020-7f17-480c-b0f4-062c82eaddd2] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:41:30.994888 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-12a07d4b-0a00-4981-9b30-6a4b6f854bcc tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:41:30.995380 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-12a07d4b-0a00-4981-9b30-6a4b6f854bcc tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:41:30.995815 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-12a07d4b-0a00-4981-9b30-6a4b6f854bcc tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:41:30.996514 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-12a07d4b-0a00-4981-9b30-6a4b6f854bcc tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:41:30.996962 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-12a07d4b-0a00-4981-9b30-6a4b6f854bcc tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:41:30.997412 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-12a07d4b-0a00-4981-9b30-6a4b6f854bcc tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:41:31.135206 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-12a07d4b-0a00-4981-9b30-6a4b6f854bcc tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3fe5a1a037fa460eb3121e4d307542da', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '79e671941c234f36b542080d256de2c6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:41:31.143504 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-12a07d4b-0a00-4981-9b30-6a4b6f854bcc tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] 10.222.0.52 "GET /compute/v2.1/servers/a29b4020-7f17-480c-b0f4-062c82eaddd2" status: 200 len: 1266 microversion: 2.1 time: 0.159670 Aug 27 02:41:31.144075 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 581/1168] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:41:30 2022] GET /compute/v2.1/servers/a29b4020-7f17-480c-b0f4-062c82eaddd2 => generated 1266 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:41:32.165045 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-c6d5745f-ecdf-466f-a5c6-1b4924748892 tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Calling method '>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:41:32.166961 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-c6d5745f-ecdf-466f-a5c6-1b4924748892 tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] [instance: a29b4020-7f17-480c-b0f4-062c82eaddd2] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:41:32.173534 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-c6d5745f-ecdf-466f-a5c6-1b4924748892 tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:41:32.174144 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-c6d5745f-ecdf-466f-a5c6-1b4924748892 tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:41:32.174546 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-c6d5745f-ecdf-466f-a5c6-1b4924748892 tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:41:32.175470 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-c6d5745f-ecdf-466f-a5c6-1b4924748892 tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:41:32.175944 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-c6d5745f-ecdf-466f-a5c6-1b4924748892 tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:41:32.176460 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-c6d5745f-ecdf-466f-a5c6-1b4924748892 tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:41:32.660887 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-c6d5745f-ecdf-466f-a5c6-1b4924748892 tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3fe5a1a037fa460eb3121e4d307542da', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '79e671941c234f36b542080d256de2c6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:41:32.670077 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-c6d5745f-ecdf-466f-a5c6-1b4924748892 tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] 10.222.0.52 "GET /compute/v2.1/servers/a29b4020-7f17-480c-b0f4-062c82eaddd2" status: 200 len: 1266 microversion: 2.1 time: 0.509567 Aug 27 02:41:32.671164 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 588/1169] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:41:32 2022] GET /compute/v2.1/servers/a29b4020-7f17-480c-b0f4-062c82eaddd2 => generated 1266 bytes in 511 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:41:32.726726 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-e64d041a-3d60-4f33-891a-06343f461b37 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:41:32.727303 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-e64d041a-3d60-4f33-891a-06343f461b37 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] [instance: 7f0ec822-1c41-4508-8d13-a48c4c88f133] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:41:32.733659 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-e64d041a-3d60-4f33-891a-06343f461b37 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:41:32.734151 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-e64d041a-3d60-4f33-891a-06343f461b37 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:41:32.734590 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-e64d041a-3d60-4f33-891a-06343f461b37 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:41:32.735289 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-e64d041a-3d60-4f33-891a-06343f461b37 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:41:32.735739 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-e64d041a-3d60-4f33-891a-06343f461b37 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:41:32.736196 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-e64d041a-3d60-4f33-891a-06343f461b37 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:41:32.761479 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-e64d041a-3d60-4f33-891a-06343f461b37 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] 10.222.0.52 "GET /compute/v2.1/servers/7f0ec822-1c41-4508-8d13-a48c4c88f133/ips" status: 200 len: 145 microversion: 2.1 time: 0.096527 Aug 27 02:41:32.762018 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 582/1170] 10.222.0.52 () {58 vars in 1248 bytes} [Sat Aug 27 02:41:32 2022] GET /compute/v2.1/servers/7f0ec822-1c41-4508-8d13-a48c4c88f133/ips => generated 145 bytes in 98 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 27 02:41:33.695274 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-1cd16637-c204-47f2-80fe-573b02d6c437 tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Calling method '>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:41:33.695902 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-1cd16637-c204-47f2-80fe-573b02d6c437 tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] [instance: a29b4020-7f17-480c-b0f4-062c82eaddd2] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:41:33.704531 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-1cd16637-c204-47f2-80fe-573b02d6c437 tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:41:33.705274 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-1cd16637-c204-47f2-80fe-573b02d6c437 tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:41:33.705885 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-1cd16637-c204-47f2-80fe-573b02d6c437 tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:41:33.706744 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-1cd16637-c204-47f2-80fe-573b02d6c437 tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:41:33.707469 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-1cd16637-c204-47f2-80fe-573b02d6c437 tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:41:33.708045 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-1cd16637-c204-47f2-80fe-573b02d6c437 tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:41:33.856653 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-1cd16637-c204-47f2-80fe-573b02d6c437 tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3fe5a1a037fa460eb3121e4d307542da', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '79e671941c234f36b542080d256de2c6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:41:33.865858 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-1cd16637-c204-47f2-80fe-573b02d6c437 tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] 10.222.0.52 "GET /compute/v2.1/servers/a29b4020-7f17-480c-b0f4-062c82eaddd2" status: 200 len: 1266 microversion: 2.1 time: 0.176875 Aug 27 02:41:33.866427 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 589/1171] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:41:33 2022] GET /compute/v2.1/servers/a29b4020-7f17-480c-b0f4-062c82eaddd2 => generated 1266 bytes in 179 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:41:34.126255 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-b9d4830b-1157-4c74-9564-8096387dbec2 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "68000d21-3712-4ea6-b2d4-5e563c683ec1"}} {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 27 02:41:34.128688 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-b9d4830b-1157-4c74-9564-8096387dbec2 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] [instance: 7f0ec822-1c41-4508-8d13-a48c4c88f133] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:41:34.135826 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-b9d4830b-1157-4c74-9564-8096387dbec2 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:41:34.136318 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-b9d4830b-1157-4c74-9564-8096387dbec2 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:41:34.136754 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-b9d4830b-1157-4c74-9564-8096387dbec2 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:41:34.137481 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-b9d4830b-1157-4c74-9564-8096387dbec2 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:41:34.137926 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-b9d4830b-1157-4c74-9564-8096387dbec2 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:41:34.138355 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-b9d4830b-1157-4c74-9564-8096387dbec2 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:41:34.449835 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-b9d4830b-1157-4c74-9564-8096387dbec2 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] 10.222.0.52 "POST /compute/v2.1/servers/7f0ec822-1c41-4508-8d13-a48c4c88f133/os-volume_attachments" status: 200 len: 194 microversion: 2.1 time: 0.327365 Aug 27 02:41:34.450446 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 583/1172] 10.222.0.52 () {60 vars in 1323 bytes} [Sat Aug 27 02:41:34 2022] POST /compute/v2.1/servers/7f0ec822-1c41-4508-8d13-a48c4c88f133/os-volume_attachments => generated 194 bytes in 329 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 27 02:41:34.889248 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-5ddcbc02-2b24-4564-b90b-3940f146fe75 tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Calling method '>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:41:34.889812 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-5ddcbc02-2b24-4564-b90b-3940f146fe75 tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] [instance: a29b4020-7f17-480c-b0f4-062c82eaddd2] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:41:34.896274 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-5ddcbc02-2b24-4564-b90b-3940f146fe75 tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:41:34.896761 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-5ddcbc02-2b24-4564-b90b-3940f146fe75 tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:41:34.897221 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-5ddcbc02-2b24-4564-b90b-3940f146fe75 tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:41:34.897945 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-5ddcbc02-2b24-4564-b90b-3940f146fe75 tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:41:34.898394 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-5ddcbc02-2b24-4564-b90b-3940f146fe75 tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:41:34.898815 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-5ddcbc02-2b24-4564-b90b-3940f146fe75 tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:41:35.043768 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-5ddcbc02-2b24-4564-b90b-3940f146fe75 tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3fe5a1a037fa460eb3121e4d307542da', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '79e671941c234f36b542080d256de2c6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:41:35.051378 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-5ddcbc02-2b24-4564-b90b-3940f146fe75 tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] 10.222.0.52 "GET /compute/v2.1/servers/a29b4020-7f17-480c-b0f4-062c82eaddd2" status: 200 len: 1266 microversion: 2.1 time: 0.166141 Aug 27 02:41:35.051938 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 590/1173] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:41:34 2022] GET /compute/v2.1/servers/a29b4020-7f17-480c-b0f4-062c82eaddd2 => generated 1266 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:41:36.076309 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-1afd10eb-9088-4124-a8ae-64cd47a94154 tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:41:36.077045 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-1afd10eb-9088-4124-a8ae-64cd47a94154 tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] [instance: a29b4020-7f17-480c-b0f4-062c82eaddd2] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:41:36.084332 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-1afd10eb-9088-4124-a8ae-64cd47a94154 tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:41:36.084870 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-1afd10eb-9088-4124-a8ae-64cd47a94154 tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:41:36.085439 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-1afd10eb-9088-4124-a8ae-64cd47a94154 tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:41:36.086204 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-1afd10eb-9088-4124-a8ae-64cd47a94154 tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:41:36.086709 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-1afd10eb-9088-4124-a8ae-64cd47a94154 tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:41:36.088018 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-1afd10eb-9088-4124-a8ae-64cd47a94154 tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:41:36.280701 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-1afd10eb-9088-4124-a8ae-64cd47a94154 tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3fe5a1a037fa460eb3121e4d307542da', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '79e671941c234f36b542080d256de2c6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:41:36.288688 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-1afd10eb-9088-4124-a8ae-64cd47a94154 tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] 10.222.0.52 "GET /compute/v2.1/servers/a29b4020-7f17-480c-b0f4-062c82eaddd2" status: 200 len: 1266 microversion: 2.1 time: 0.218627 Aug 27 02:41:36.289280 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 584/1174] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:41:36 2022] GET /compute/v2.1/servers/a29b4020-7f17-480c-b0f4-062c82eaddd2 => generated 1266 bytes in 220 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:41:37.310956 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-239a4a19-3167-4026-99f1-90f22582a2fb tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Calling method '>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:41:37.311616 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-239a4a19-3167-4026-99f1-90f22582a2fb tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] [instance: a29b4020-7f17-480c-b0f4-062c82eaddd2] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:41:37.314253 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG dbcounter [-] [120864] Writing DB stats nova_cell0:SELECT=2 {{(pid=120864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 27 02:41:37.314911 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG dbcounter [-] [120864] Writing DB stats nova_cell0:SELECT=6 {{(pid=120864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 27 02:41:37.320896 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-239a4a19-3167-4026-99f1-90f22582a2fb tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:41:37.321475 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-239a4a19-3167-4026-99f1-90f22582a2fb tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:41:37.321994 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-239a4a19-3167-4026-99f1-90f22582a2fb tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:41:37.323146 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-239a4a19-3167-4026-99f1-90f22582a2fb tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:41:37.323146 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-239a4a19-3167-4026-99f1-90f22582a2fb tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:41:37.323538 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-239a4a19-3167-4026-99f1-90f22582a2fb tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:41:37.480461 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-239a4a19-3167-4026-99f1-90f22582a2fb tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3fe5a1a037fa460eb3121e4d307542da', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '79e671941c234f36b542080d256de2c6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:41:37.489359 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-239a4a19-3167-4026-99f1-90f22582a2fb tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] 10.222.0.52 "GET /compute/v2.1/servers/a29b4020-7f17-480c-b0f4-062c82eaddd2" status: 200 len: 1266 microversion: 2.1 time: 0.182017 Aug 27 02:41:37.490086 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 591/1175] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:41:37 2022] GET /compute/v2.1/servers/a29b4020-7f17-480c-b0f4-062c82eaddd2 => generated 1266 bytes in 184 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:41:38.519139 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-64e25823-c15b-4d75-899a-600d77b03134 tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:41:38.520029 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-64e25823-c15b-4d75-899a-600d77b03134 tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] [instance: a29b4020-7f17-480c-b0f4-062c82eaddd2] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:41:38.523595 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG dbcounter [-] [120863] Writing DB stats nova_cell0:SELECT=2 {{(pid=120863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 27 02:41:38.532492 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-64e25823-c15b-4d75-899a-600d77b03134 tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:41:38.533260 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-64e25823-c15b-4d75-899a-600d77b03134 tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:41:38.533929 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-64e25823-c15b-4d75-899a-600d77b03134 tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:41:38.535013 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-64e25823-c15b-4d75-899a-600d77b03134 tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:41:38.535679 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-64e25823-c15b-4d75-899a-600d77b03134 tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:41:38.536327 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-64e25823-c15b-4d75-899a-600d77b03134 tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:41:38.676946 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-64e25823-c15b-4d75-899a-600d77b03134 tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3fe5a1a037fa460eb3121e4d307542da', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '79e671941c234f36b542080d256de2c6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:41:38.684996 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-64e25823-c15b-4d75-899a-600d77b03134 tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] 10.222.0.52 "GET /compute/v2.1/servers/a29b4020-7f17-480c-b0f4-062c82eaddd2" status: 200 len: 1266 microversion: 2.1 time: 0.171880 Aug 27 02:41:38.685604 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 585/1176] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:41:38 2022] GET /compute/v2.1/servers/a29b4020-7f17-480c-b0f4-062c82eaddd2 => generated 1266 bytes in 173 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:41:39.710205 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-867519f2-9f72-433b-89bb-6b07ff214bca tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Calling method '>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:41:39.710837 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-867519f2-9f72-433b-89bb-6b07ff214bca tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] [instance: a29b4020-7f17-480c-b0f4-062c82eaddd2] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:41:39.718975 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-867519f2-9f72-433b-89bb-6b07ff214bca tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:41:39.719463 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-867519f2-9f72-433b-89bb-6b07ff214bca tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:41:39.719894 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-867519f2-9f72-433b-89bb-6b07ff214bca tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:41:39.720590 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-867519f2-9f72-433b-89bb-6b07ff214bca tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:41:39.721043 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-867519f2-9f72-433b-89bb-6b07ff214bca tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:41:39.721510 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-867519f2-9f72-433b-89bb-6b07ff214bca tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:41:39.869724 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-867519f2-9f72-433b-89bb-6b07ff214bca tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3fe5a1a037fa460eb3121e4d307542da', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '79e671941c234f36b542080d256de2c6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:41:39.877784 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-867519f2-9f72-433b-89bb-6b07ff214bca tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] 10.222.0.52 "GET /compute/v2.1/servers/a29b4020-7f17-480c-b0f4-062c82eaddd2" status: 200 len: 1266 microversion: 2.1 time: 0.174346 Aug 27 02:41:39.878319 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 592/1177] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:41:39 2022] GET /compute/v2.1/servers/a29b4020-7f17-480c-b0f4-062c82eaddd2 => generated 1266 bytes in 176 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:41:40.897681 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-73ed64d6-a2c4-4616-845a-a5518457929d tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:41:40.898887 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-73ed64d6-a2c4-4616-845a-a5518457929d tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] [instance: a29b4020-7f17-480c-b0f4-062c82eaddd2] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:41:40.905528 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-73ed64d6-a2c4-4616-845a-a5518457929d tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:41:40.906006 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-73ed64d6-a2c4-4616-845a-a5518457929d tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:41:40.906440 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-73ed64d6-a2c4-4616-845a-a5518457929d tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:41:40.907113 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-73ed64d6-a2c4-4616-845a-a5518457929d tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:41:40.907568 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-73ed64d6-a2c4-4616-845a-a5518457929d tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:41:40.907994 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-73ed64d6-a2c4-4616-845a-a5518457929d tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:41:41.046319 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-73ed64d6-a2c4-4616-845a-a5518457929d tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3fe5a1a037fa460eb3121e4d307542da', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '79e671941c234f36b542080d256de2c6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:41:41.057736 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-73ed64d6-a2c4-4616-845a-a5518457929d tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] 10.222.0.52 "GET /compute/v2.1/servers/a29b4020-7f17-480c-b0f4-062c82eaddd2" status: 200 len: 1266 microversion: 2.1 time: 0.163948 Aug 27 02:41:41.058312 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 586/1178] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:41:40 2022] GET /compute/v2.1/servers/a29b4020-7f17-480c-b0f4-062c82eaddd2 => generated 1266 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:41:42.051401 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-937a0ce8-26dd-49eb-a631-82da98ea82ae tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Calling method '>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:41:42.053791 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-937a0ce8-26dd-49eb-a631-82da98ea82ae tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] [instance: 7f0ec822-1c41-4508-8d13-a48c4c88f133] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:41:42.060851 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-937a0ce8-26dd-49eb-a631-82da98ea82ae tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:41:42.061369 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-937a0ce8-26dd-49eb-a631-82da98ea82ae tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:41:42.061828 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-937a0ce8-26dd-49eb-a631-82da98ea82ae tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:41:42.062825 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-937a0ce8-26dd-49eb-a631-82da98ea82ae tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:41:42.063276 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-937a0ce8-26dd-49eb-a631-82da98ea82ae tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:41:42.063686 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-937a0ce8-26dd-49eb-a631-82da98ea82ae tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:41:42.078664 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-5329fed2-1c55-436b-894d-f376ddeda6f3 tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:41:42.079352 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-5329fed2-1c55-436b-894d-f376ddeda6f3 tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] [instance: a29b4020-7f17-480c-b0f4-062c82eaddd2] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:41:42.087620 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-5329fed2-1c55-436b-894d-f376ddeda6f3 tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:41:42.088128 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-5329fed2-1c55-436b-894d-f376ddeda6f3 tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:41:42.088567 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-5329fed2-1c55-436b-894d-f376ddeda6f3 tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:41:42.089308 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-5329fed2-1c55-436b-894d-f376ddeda6f3 tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:41:42.089932 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-5329fed2-1c55-436b-894d-f376ddeda6f3 tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:41:42.090549 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-5329fed2-1c55-436b-894d-f376ddeda6f3 tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:41:42.098569 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-937a0ce8-26dd-49eb-a631-82da98ea82ae tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] 10.222.0.52 "GET /compute/v2.1/servers/7f0ec822-1c41-4508-8d13-a48c4c88f133/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.051201 Aug 27 02:41:42.099137 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 593/1179] 10.222.0.52 () {58 vars in 1302 bytes} [Sat Aug 27 02:41:42 2022] GET /compute/v2.1/servers/7f0ec822-1c41-4508-8d13-a48c4c88f133/os-volume_attachments => generated 197 bytes in 52 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Aug 27 02:41:42.109392 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-39189acf-0e09-4d9d-8786-717323328857 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Calling method '>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:41:42.109987 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-39189acf-0e09-4d9d-8786-717323328857 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] [instance: 7f0ec822-1c41-4508-8d13-a48c4c88f133] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:41:42.115940 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-39189acf-0e09-4d9d-8786-717323328857 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:41:42.116423 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-39189acf-0e09-4d9d-8786-717323328857 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:41:42.116859 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-39189acf-0e09-4d9d-8786-717323328857 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:41:42.117573 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-39189acf-0e09-4d9d-8786-717323328857 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:41:42.118176 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-39189acf-0e09-4d9d-8786-717323328857 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:41:42.118509 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-39189acf-0e09-4d9d-8786-717323328857 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:41:42.148916 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-39189acf-0e09-4d9d-8786-717323328857 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] 10.222.0.52 "GET /compute/v2.1/servers/7f0ec822-1c41-4508-8d13-a48c4c88f133/os-volume_attachments/68000d21-3712-4ea6-b2d4-5e563c683ec1" status: 200 len: 194 microversion: 2.1 time: 0.043158 Aug 27 02:41:42.150331 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 594/1180] 10.222.0.52 () {58 vars in 1413 bytes} [Sat Aug 27 02:41:42 2022] GET /compute/v2.1/servers/7f0ec822-1c41-4508-8d13-a48c4c88f133/os-volume_attachments/68000d21-3712-4ea6-b2d4-5e563c683ec1 => generated 194 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Aug 27 02:41:42.228375 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-5329fed2-1c55-436b-894d-f376ddeda6f3 tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3fe5a1a037fa460eb3121e4d307542da', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '79e671941c234f36b542080d256de2c6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:41:42.236463 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-5329fed2-1c55-436b-894d-f376ddeda6f3 tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] 10.222.0.52 "GET /compute/v2.1/servers/a29b4020-7f17-480c-b0f4-062c82eaddd2" status: 200 len: 1266 microversion: 2.1 time: 0.161717 Aug 27 02:41:42.237139 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 587/1181] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:41:42 2022] GET /compute/v2.1/servers/a29b4020-7f17-480c-b0f4-062c82eaddd2 => generated 1266 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 27 02:41:42.598989 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [req-3899662f-f5b4-4e3d-8a82-1be8fa4a3578 req-a51d113a-dae6-4b8a-9fd1-1535fd05cda8 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "a29b4020-7f17-480c-b0f4-062c82eaddd2", "name": "network-vif-plugged", "status": "completed", "tag": "0f6ab26e-c941-4e20-b27e-ed8630548227"}]} {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 27 02:41:42.609605 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [req-3899662f-f5b4-4e3d-8a82-1be8fa4a3578 req-a51d113a-dae6-4b8a-9fd1-1535fd05cda8 service nova] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:41:42.610095 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [req-3899662f-f5b4-4e3d-8a82-1be8fa4a3578 req-a51d113a-dae6-4b8a-9fd1-1535fd05cda8 service nova] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:41:42.610644 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [req-3899662f-f5b4-4e3d-8a82-1be8fa4a3578 req-a51d113a-dae6-4b8a-9fd1-1535fd05cda8 service nova] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:41:42.622674 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.compute.server_external_events [req-3899662f-f5b4-4e3d-8a82-1be8fa4a3578 req-a51d113a-dae6-4b8a-9fd1-1535fd05cda8 service nova] Creating event network-vif-plugged:0f6ab26e-c941-4e20-b27e-ed8630548227 for instance a29b4020-7f17-480c-b0f4-062c82eaddd2 on hv-cinder-84630 Aug 27 02:41:42.627770 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [req-3899662f-f5b4-4e3d-8a82-1be8fa4a3578 req-a51d113a-dae6-4b8a-9fd1-1535fd05cda8 service nova] 10.222.0.52 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.032490 Aug 27 02:41:42.628394 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 595/1182] 10.222.0.52 () {64 vars in 1317 bytes} [Sat Aug 27 02:41:42 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 34 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 27 02:41:43.260410 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-0cf2be81-2b2c-4eb1-ac37-75c1e796c576 tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:41:43.260987 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-0cf2be81-2b2c-4eb1-ac37-75c1e796c576 tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] [instance: a29b4020-7f17-480c-b0f4-062c82eaddd2] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:41:43.268151 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-0cf2be81-2b2c-4eb1-ac37-75c1e796c576 tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:41:43.268638 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-0cf2be81-2b2c-4eb1-ac37-75c1e796c576 tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:41:43.269074 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-0cf2be81-2b2c-4eb1-ac37-75c1e796c576 tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:41:43.269797 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-0cf2be81-2b2c-4eb1-ac37-75c1e796c576 tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:41:43.270252 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-0cf2be81-2b2c-4eb1-ac37-75c1e796c576 tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:41:43.270677 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-0cf2be81-2b2c-4eb1-ac37-75c1e796c576 tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:41:43.415640 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-0cf2be81-2b2c-4eb1-ac37-75c1e796c576 tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3fe5a1a037fa460eb3121e4d307542da', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '79e671941c234f36b542080d256de2c6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:41:43.427098 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-0cf2be81-2b2c-4eb1-ac37-75c1e796c576 tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] 10.222.0.52 "GET /compute/v2.1/servers/a29b4020-7f17-480c-b0f4-062c82eaddd2" status: 200 len: 1266 microversion: 2.1 time: 0.172029 Aug 27 02:41:43.427747 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 588/1183] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:41:43 2022] GET /compute/v2.1/servers/a29b4020-7f17-480c-b0f4-062c82eaddd2 => generated 1266 bytes in 174 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:41:43.525875 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-f9f19486-1b52-4fd7-8059-4fdabe58706c tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "b3134075-838e-4b36-9d16-020ad9f62b8b"}} {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 27 02:41:43.528241 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-f9f19486-1b52-4fd7-8059-4fdabe58706c tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] [instance: 7f0ec822-1c41-4508-8d13-a48c4c88f133] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:41:43.534867 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-f9f19486-1b52-4fd7-8059-4fdabe58706c tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:41:43.535350 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-f9f19486-1b52-4fd7-8059-4fdabe58706c tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:41:43.535787 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-f9f19486-1b52-4fd7-8059-4fdabe58706c tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:41:43.536479 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-f9f19486-1b52-4fd7-8059-4fdabe58706c tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:41:43.536925 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-f9f19486-1b52-4fd7-8059-4fdabe58706c tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:41:43.537383 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-f9f19486-1b52-4fd7-8059-4fdabe58706c tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:41:44.345228 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-f9f19486-1b52-4fd7-8059-4fdabe58706c tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] 10.222.0.52 "POST /compute/v2.1/servers/7f0ec822-1c41-4508-8d13-a48c4c88f133/os-volume_attachments" status: 200 len: 194 microversion: 2.1 time: 0.824302 Aug 27 02:41:44.345842 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 596/1184] 10.222.0.52 () {60 vars in 1323 bytes} [Sat Aug 27 02:41:43 2022] POST /compute/v2.1/servers/7f0ec822-1c41-4508-8d13-a48c4c88f133/os-volume_attachments => generated 194 bytes in 826 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 27 02:41:44.449346 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-2ee5f09d-43e9-431e-b8c4-24e2eeeba7e3 tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:41:44.449954 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-2ee5f09d-43e9-431e-b8c4-24e2eeeba7e3 tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] [instance: a29b4020-7f17-480c-b0f4-062c82eaddd2] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:41:44.457175 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-2ee5f09d-43e9-431e-b8c4-24e2eeeba7e3 tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:41:44.457683 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-2ee5f09d-43e9-431e-b8c4-24e2eeeba7e3 tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:41:44.458263 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-2ee5f09d-43e9-431e-b8c4-24e2eeeba7e3 tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:41:44.458993 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-2ee5f09d-43e9-431e-b8c4-24e2eeeba7e3 tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:41:44.459423 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-2ee5f09d-43e9-431e-b8c4-24e2eeeba7e3 tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:41:44.459850 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-2ee5f09d-43e9-431e-b8c4-24e2eeeba7e3 tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:41:44.605348 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-2ee5f09d-43e9-431e-b8c4-24e2eeeba7e3 tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3fe5a1a037fa460eb3121e4d307542da', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '79e671941c234f36b542080d256de2c6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:41:44.613534 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-2ee5f09d-43e9-431e-b8c4-24e2eeeba7e3 tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] 10.222.0.52 "GET /compute/v2.1/servers/a29b4020-7f17-480c-b0f4-062c82eaddd2" status: 200 len: 1266 microversion: 2.1 time: 0.168125 Aug 27 02:41:44.614115 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 589/1185] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:41:44 2022] GET /compute/v2.1/servers/a29b4020-7f17-480c-b0f4-062c82eaddd2 => generated 1266 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:41:45.636834 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-1dc31ce1-73e7-4e99-aea1-7b50f4a0d6a4 tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Calling method '>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:41:45.637498 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-1dc31ce1-73e7-4e99-aea1-7b50f4a0d6a4 tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] [instance: a29b4020-7f17-480c-b0f4-062c82eaddd2] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:41:45.644619 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-1dc31ce1-73e7-4e99-aea1-7b50f4a0d6a4 tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:41:45.645098 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-1dc31ce1-73e7-4e99-aea1-7b50f4a0d6a4 tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:41:45.645562 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-1dc31ce1-73e7-4e99-aea1-7b50f4a0d6a4 tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:41:45.646226 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-1dc31ce1-73e7-4e99-aea1-7b50f4a0d6a4 tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:41:45.646669 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-1dc31ce1-73e7-4e99-aea1-7b50f4a0d6a4 tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:41:45.647081 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-1dc31ce1-73e7-4e99-aea1-7b50f4a0d6a4 tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:41:45.794746 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-1dc31ce1-73e7-4e99-aea1-7b50f4a0d6a4 tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3fe5a1a037fa460eb3121e4d307542da', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '79e671941c234f36b542080d256de2c6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:41:45.802946 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-1dc31ce1-73e7-4e99-aea1-7b50f4a0d6a4 tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] 10.222.0.52 "GET /compute/v2.1/servers/a29b4020-7f17-480c-b0f4-062c82eaddd2" status: 200 len: 1453 microversion: 2.1 time: 0.171029 Aug 27 02:41:45.803509 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 597/1186] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:41:45 2022] GET /compute/v2.1/servers/a29b4020-7f17-480c-b0f4-062c82eaddd2 => generated 1453 bytes in 172 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:41:45.823924 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-6016229f-4073-4480-80ca-57fd93aff730 tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:41:45.824516 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-6016229f-4073-4480-80ca-57fd93aff730 tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] [instance: a29b4020-7f17-480c-b0f4-062c82eaddd2] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:41:45.831173 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-6016229f-4073-4480-80ca-57fd93aff730 tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:41:45.831652 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-6016229f-4073-4480-80ca-57fd93aff730 tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:41:45.832088 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-6016229f-4073-4480-80ca-57fd93aff730 tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:41:45.832775 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-6016229f-4073-4480-80ca-57fd93aff730 tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:41:45.833261 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-6016229f-4073-4480-80ca-57fd93aff730 tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:41:45.833679 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-6016229f-4073-4480-80ca-57fd93aff730 tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:41:45.984707 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-6016229f-4073-4480-80ca-57fd93aff730 tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3fe5a1a037fa460eb3121e4d307542da', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '79e671941c234f36b542080d256de2c6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:41:45.993384 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-6016229f-4073-4480-80ca-57fd93aff730 tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] 10.222.0.52 "GET /compute/v2.1/servers/a29b4020-7f17-480c-b0f4-062c82eaddd2" status: 200 len: 1453 microversion: 2.1 time: 0.173276 Aug 27 02:41:45.993964 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 590/1187] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:41:45 2022] GET /compute/v2.1/servers/a29b4020-7f17-480c-b0f4-062c82eaddd2 => generated 1453 bytes in 175 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:41:47.790004 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [req-f514f266-cbb2-428a-bd37-fdc731b39e3d req-5c78a8fe-9d37-45fc-aa90-d3293cf14833 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "a29b4020-7f17-480c-b0f4-062c82eaddd2", "tag": "0f6ab26e-c941-4e20-b27e-ed8630548227"}]} {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 27 02:41:47.800209 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [req-f514f266-cbb2-428a-bd37-fdc731b39e3d req-5c78a8fe-9d37-45fc-aa90-d3293cf14833 service nova] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:41:47.800686 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [req-f514f266-cbb2-428a-bd37-fdc731b39e3d req-5c78a8fe-9d37-45fc-aa90-d3293cf14833 service nova] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:41:47.801138 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [req-f514f266-cbb2-428a-bd37-fdc731b39e3d req-5c78a8fe-9d37-45fc-aa90-d3293cf14833 service nova] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:41:47.812948 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.compute.server_external_events [req-f514f266-cbb2-428a-bd37-fdc731b39e3d req-5c78a8fe-9d37-45fc-aa90-d3293cf14833 service nova] Creating event network-changed:0f6ab26e-c941-4e20-b27e-ed8630548227 for instance a29b4020-7f17-480c-b0f4-062c82eaddd2 on hv-cinder-84630 Aug 27 02:41:47.817957 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [req-f514f266-cbb2-428a-bd37-fdc731b39e3d req-5c78a8fe-9d37-45fc-aa90-d3293cf14833 service nova] 10.222.0.52 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.031839 Aug 27 02:41:47.818560 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 598/1188] 10.222.0.52 () {64 vars in 1317 bytes} [Sat Aug 27 02:41:47 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 33 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 27 02:41:51.902658 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-3f62e11d-132d-44f3-8842-df2e631d2933 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:41:51.904915 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-3f62e11d-132d-44f3-8842-df2e631d2933 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] [instance: 7f0ec822-1c41-4508-8d13-a48c4c88f133] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:41:51.911744 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-3f62e11d-132d-44f3-8842-df2e631d2933 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:41:51.912227 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-3f62e11d-132d-44f3-8842-df2e631d2933 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:41:51.912655 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-3f62e11d-132d-44f3-8842-df2e631d2933 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:41:51.913376 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-3f62e11d-132d-44f3-8842-df2e631d2933 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:41:51.913823 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-3f62e11d-132d-44f3-8842-df2e631d2933 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:41:51.914531 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-3f62e11d-132d-44f3-8842-df2e631d2933 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:41:51.946052 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-3f62e11d-132d-44f3-8842-df2e631d2933 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] 10.222.0.52 "GET /compute/v2.1/servers/7f0ec822-1c41-4508-8d13-a48c4c88f133/os-volume_attachments" status: 200 len: 371 microversion: 2.1 time: 0.047155 Aug 27 02:41:51.946775 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 591/1189] 10.222.0.52 () {58 vars in 1302 bytes} [Sat Aug 27 02:41:51 2022] GET /compute/v2.1/servers/7f0ec822-1c41-4508-8d13-a48c4c88f133/os-volume_attachments => generated 371 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 27 02:41:52.088926 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-b86b3cb2-d857-4fb5-b5de-31b70f8cd062 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Calling method '>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:41:52.089756 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-b86b3cb2-d857-4fb5-b5de-31b70f8cd062 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] [instance: 7f0ec822-1c41-4508-8d13-a48c4c88f133] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:41:52.096790 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-b86b3cb2-d857-4fb5-b5de-31b70f8cd062 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:41:52.097306 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-b86b3cb2-d857-4fb5-b5de-31b70f8cd062 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:41:52.097737 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-b86b3cb2-d857-4fb5-b5de-31b70f8cd062 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:41:52.098437 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-b86b3cb2-d857-4fb5-b5de-31b70f8cd062 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:41:52.098885 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-b86b3cb2-d857-4fb5-b5de-31b70f8cd062 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:41:52.099309 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-b86b3cb2-d857-4fb5-b5de-31b70f8cd062 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:41:52.130290 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-b86b3cb2-d857-4fb5-b5de-31b70f8cd062 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] 10.222.0.52 "GET /compute/v2.1/servers/7f0ec822-1c41-4508-8d13-a48c4c88f133/os-volume_attachments/68000d21-3712-4ea6-b2d4-5e563c683ec1" status: 200 len: 194 microversion: 2.1 time: 0.046290 Aug 27 02:41:52.130893 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 599/1190] 10.222.0.52 () {58 vars in 1413 bytes} [Sat Aug 27 02:41:52 2022] GET /compute/v2.1/servers/7f0ec822-1c41-4508-8d13-a48c4c88f133/os-volume_attachments/68000d21-3712-4ea6-b2d4-5e563c683ec1 => generated 194 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 27 02:41:52.146390 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-7285556e-ca0a-4ff8-aa69-d26ee9d7cea4 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:41:52.147105 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-7285556e-ca0a-4ff8-aa69-d26ee9d7cea4 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] [instance: 7f0ec822-1c41-4508-8d13-a48c4c88f133] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:41:52.155553 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-7285556e-ca0a-4ff8-aa69-d26ee9d7cea4 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:41:52.156030 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-7285556e-ca0a-4ff8-aa69-d26ee9d7cea4 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:41:52.156657 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-7285556e-ca0a-4ff8-aa69-d26ee9d7cea4 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:41:52.157389 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-7285556e-ca0a-4ff8-aa69-d26ee9d7cea4 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:41:52.157839 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-7285556e-ca0a-4ff8-aa69-d26ee9d7cea4 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:41:52.158277 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-7285556e-ca0a-4ff8-aa69-d26ee9d7cea4 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:41:52.246544 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.objects.instance [None req-7285556e-ca0a-4ff8-aa69-d26ee9d7cea4 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Lazy-loading 'services' on Instance uuid 7f0ec822-1c41-4508-8d13-a48c4c88f133 {{(pid=120863) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Aug 27 02:41:52.332101 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-7285556e-ca0a-4ff8-aa69-d26ee9d7cea4 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] 10.222.0.52 "DELETE /compute/v2.1/servers/7f0ec822-1c41-4508-8d13-a48c4c88f133/os-volume_attachments/68000d21-3712-4ea6-b2d4-5e563c683ec1" status: 202 len: 0 microversion: 2.1 time: 0.191381 Aug 27 02:41:52.332723 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 592/1191] 10.222.0.52 () {58 vars in 1416 bytes} [Sat Aug 27 02:41:52 2022] DELETE /compute/v2.1/servers/7f0ec822-1c41-4508-8d13-a48c4c88f133/os-volume_attachments/68000d21-3712-4ea6-b2d4-5e563c683ec1 => generated 0 bytes in 194 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Aug 27 02:41:54.520361 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-4e82686b-1462-488e-8f6a-98105efc88ac tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Calling method '>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:41:54.520920 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-4e82686b-1462-488e-8f6a-98105efc88ac tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] [instance: 7f0ec822-1c41-4508-8d13-a48c4c88f133] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:41:54.528081 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-4e82686b-1462-488e-8f6a-98105efc88ac tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:41:54.528570 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-4e82686b-1462-488e-8f6a-98105efc88ac tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:41:54.529002 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-4e82686b-1462-488e-8f6a-98105efc88ac tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:41:54.529737 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-4e82686b-1462-488e-8f6a-98105efc88ac tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:41:54.530187 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-4e82686b-1462-488e-8f6a-98105efc88ac tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:41:54.530576 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-4e82686b-1462-488e-8f6a-98105efc88ac tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:41:54.561571 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-4e82686b-1462-488e-8f6a-98105efc88ac tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] 10.222.0.52 "GET /compute/v2.1/servers/7f0ec822-1c41-4508-8d13-a48c4c88f133/os-volume_attachments/b3134075-838e-4b36-9d16-020ad9f62b8b" status: 200 len: 194 microversion: 2.1 time: 0.045150 Aug 27 02:41:54.562469 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 600/1192] 10.222.0.52 () {58 vars in 1413 bytes} [Sat Aug 27 02:41:54 2022] GET /compute/v2.1/servers/7f0ec822-1c41-4508-8d13-a48c4c88f133/os-volume_attachments/b3134075-838e-4b36-9d16-020ad9f62b8b => generated 194 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Aug 27 02:41:54.574438 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-34193104-d9fe-42f0-a633-9e097f70e40a tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:41:54.575315 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-34193104-d9fe-42f0-a633-9e097f70e40a tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] [instance: 7f0ec822-1c41-4508-8d13-a48c4c88f133] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:41:54.582987 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-34193104-d9fe-42f0-a633-9e097f70e40a tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:41:54.583610 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-34193104-d9fe-42f0-a633-9e097f70e40a tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:41:54.584054 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-34193104-d9fe-42f0-a633-9e097f70e40a tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:41:54.584793 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-34193104-d9fe-42f0-a633-9e097f70e40a tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:41:54.585280 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-34193104-d9fe-42f0-a633-9e097f70e40a tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:41:54.586199 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-34193104-d9fe-42f0-a633-9e097f70e40a tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:41:54.671509 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.objects.instance [None req-34193104-d9fe-42f0-a633-9e097f70e40a tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Lazy-loading 'services' on Instance uuid 7f0ec822-1c41-4508-8d13-a48c4c88f133 {{(pid=120863) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Aug 27 02:41:54.745944 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-34193104-d9fe-42f0-a633-9e097f70e40a tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] 10.222.0.52 "DELETE /compute/v2.1/servers/7f0ec822-1c41-4508-8d13-a48c4c88f133/os-volume_attachments/b3134075-838e-4b36-9d16-020ad9f62b8b" status: 202 len: 0 microversion: 2.1 time: 0.176497 Aug 27 02:41:54.746570 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 593/1193] 10.222.0.52 () {58 vars in 1416 bytes} [Sat Aug 27 02:41:54 2022] DELETE /compute/v2.1/servers/7f0ec822-1c41-4508-8d13-a48c4c88f133/os-volume_attachments/b3134075-838e-4b36-9d16-020ad9f62b8b => generated 0 bytes in 178 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Aug 27 02:41:55.936202 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-382e938f-5afe-48ed-a154-2d39dcce43ce tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Calling method '>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:41:55.938722 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-382e938f-5afe-48ed-a154-2d39dcce43ce tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] [instance: 7f0ec822-1c41-4508-8d13-a48c4c88f133] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:41:55.945891 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-382e938f-5afe-48ed-a154-2d39dcce43ce tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:41:55.946489 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-382e938f-5afe-48ed-a154-2d39dcce43ce tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:41:55.947021 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-382e938f-5afe-48ed-a154-2d39dcce43ce tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:41:55.947808 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-382e938f-5afe-48ed-a154-2d39dcce43ce tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:41:55.948348 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-382e938f-5afe-48ed-a154-2d39dcce43ce tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:41:55.948932 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-382e938f-5afe-48ed-a154-2d39dcce43ce tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:41:55.980322 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-382e938f-5afe-48ed-a154-2d39dcce43ce tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] 10.222.0.52 "GET /compute/v2.1/servers/7f0ec822-1c41-4508-8d13-a48c4c88f133/os-volume_attachments" status: 200 len: 25 microversion: 2.1 time: 0.048346 Aug 27 02:41:55.980908 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 601/1194] 10.222.0.52 () {58 vars in 1302 bytes} [Sat Aug 27 02:41:55 2022] GET /compute/v2.1/servers/7f0ec822-1c41-4508-8d13-a48c4c88f133/os-volume_attachments => generated 25 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Aug 27 02:41:56.100063 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Aug 27 02:41:56.100825 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: Sat Aug 27 02:41:56 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/7f0ec822-1c41-4508-8d13-a48c4c88f133/os-volume_attachments (ip 10.222.0.52) !!! Aug 27 02:41:56.105613 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-c65c498d-f0d5-4b34-9c18-2e9d6bf52765 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:41:56.107930 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-c65c498d-f0d5-4b34-9c18-2e9d6bf52765 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] [instance: 7f0ec822-1c41-4508-8d13-a48c4c88f133] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:41:56.116325 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-c65c498d-f0d5-4b34-9c18-2e9d6bf52765 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:41:56.116937 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-c65c498d-f0d5-4b34-9c18-2e9d6bf52765 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:41:56.117421 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-c65c498d-f0d5-4b34-9c18-2e9d6bf52765 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:41:56.118289 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-c65c498d-f0d5-4b34-9c18-2e9d6bf52765 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:41:56.118854 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-c65c498d-f0d5-4b34-9c18-2e9d6bf52765 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:41:56.119360 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-c65c498d-f0d5-4b34-9c18-2e9d6bf52765 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:41:56.125200 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG dbcounter [-] [120863] Writing DB stats nova_cell1:SELECT=95,nova_cell1:UPDATE=2,nova_cell1:INSERT=3 {{(pid=120863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 27 02:41:56.154116 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-c65c498d-f0d5-4b34-9c18-2e9d6bf52765 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] 10.222.0.52 "GET /compute/v2.1/servers/7f0ec822-1c41-4508-8d13-a48c4c88f133/os-volume_attachments" status: 200 len: 25 microversion: 2.1 time: 0.055669 Aug 27 02:41:56.154584 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 594/1195] 10.222.0.52 () {58 vars in 1302 bytes} [Sat Aug 27 02:41:56 2022] GET /compute/v2.1/servers/7f0ec822-1c41-4508-8d13-a48c4c88f133/os-volume_attachments => generated 25 bytes in 58 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Aug 27 02:41:56.228529 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-6457aa25-622a-43cc-840d-87696ff997c1 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Calling method '>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:41:56.229023 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-6457aa25-622a-43cc-840d-87696ff997c1 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] [instance: 7f0ec822-1c41-4508-8d13-a48c4c88f133] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:41:56.235788 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-6457aa25-622a-43cc-840d-87696ff997c1 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:41:56.236282 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-6457aa25-622a-43cc-840d-87696ff997c1 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:41:56.236726 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-6457aa25-622a-43cc-840d-87696ff997c1 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:41:56.237469 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-6457aa25-622a-43cc-840d-87696ff997c1 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:41:56.237931 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-6457aa25-622a-43cc-840d-87696ff997c1 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:41:56.238358 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-6457aa25-622a-43cc-840d-87696ff997c1 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:41:56.267707 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-6457aa25-622a-43cc-840d-87696ff997c1 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] [instance: 7f0ec822-1c41-4508-8d13-a48c4c88f133] Going to try to terminate instance {{(pid=120864) delete /opt/stack/nova/nova/compute/api.py:2719}} Aug 27 02:41:56.270762 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG dbcounter [-] [120864] Writing DB stats nova_cell1:SELECT=97,nova_cell1:UPDATE=2,nova_cell1:INSERT=1 {{(pid=120864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 27 02:41:56.362813 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-6457aa25-622a-43cc-840d-87696ff997c1 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] 10.222.0.52 "DELETE /compute/v2.1/servers/7f0ec822-1c41-4508-8d13-a48c4c88f133" status: 204 len: 0 microversion: 2.1 time: 0.138326 Aug 27 02:41:56.363402 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 602/1196] 10.222.0.52 () {58 vars in 1239 bytes} [Sat Aug 27 02:41:56 2022] DELETE /compute/v2.1/servers/7f0ec822-1c41-4508-8d13-a48c4c88f133 => generated 0 bytes in 140 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Aug 27 02:41:56.373970 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-39558803-6515-4b39-9593-34f9e6f61969 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:41:56.374579 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-39558803-6515-4b39-9593-34f9e6f61969 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] [instance: 7f0ec822-1c41-4508-8d13-a48c4c88f133] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:41:56.380536 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-39558803-6515-4b39-9593-34f9e6f61969 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:41:56.381011 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-39558803-6515-4b39-9593-34f9e6f61969 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:41:56.381501 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-39558803-6515-4b39-9593-34f9e6f61969 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:41:56.382185 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-39558803-6515-4b39-9593-34f9e6f61969 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:41:56.383164 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-39558803-6515-4b39-9593-34f9e6f61969 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:41:56.383597 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-39558803-6515-4b39-9593-34f9e6f61969 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:41:56.531098 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-39558803-6515-4b39-9593-34f9e6f61969 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee6084fffc544dfe8f1a9a30ae06be69', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a8055f4326f640fc8927f5cfdef39dd3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:41:56.542122 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-39558803-6515-4b39-9593-34f9e6f61969 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] 10.222.0.52 "GET /compute/v2.1/servers/7f0ec822-1c41-4508-8d13-a48c4c88f133" status: 200 len: 1673 microversion: 2.1 time: 0.171808 Aug 27 02:41:56.542778 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 595/1197] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:41:56 2022] GET /compute/v2.1/servers/7f0ec822-1c41-4508-8d13-a48c4c88f133 => generated 1673 bytes in 173 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:41:57.570146 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-2854e35e-edc8-4644-a10d-d77e439d3578 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Calling method '>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:41:57.571072 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-2854e35e-edc8-4644-a10d-d77e439d3578 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] [instance: 7f0ec822-1c41-4508-8d13-a48c4c88f133] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:41:57.575216 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG dbcounter [-] [120864] Writing DB stats nova_api:SELECT=55,nova_api:INSERT=3,nova_api:UPDATE=1 {{(pid=120864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 27 02:41:57.585636 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-2854e35e-edc8-4644-a10d-d77e439d3578 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:41:57.586347 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-2854e35e-edc8-4644-a10d-d77e439d3578 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:41:57.587009 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-2854e35e-edc8-4644-a10d-d77e439d3578 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:41:57.587987 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-2854e35e-edc8-4644-a10d-d77e439d3578 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:41:57.588538 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-2854e35e-edc8-4644-a10d-d77e439d3578 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:41:57.589155 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-2854e35e-edc8-4644-a10d-d77e439d3578 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:41:57.738586 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-2854e35e-edc8-4644-a10d-d77e439d3578 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee6084fffc544dfe8f1a9a30ae06be69', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a8055f4326f640fc8927f5cfdef39dd3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:41:57.746737 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-2854e35e-edc8-4644-a10d-d77e439d3578 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] 10.222.0.52 "GET /compute/v2.1/servers/7f0ec822-1c41-4508-8d13-a48c4c88f133" status: 200 len: 1673 microversion: 2.1 time: 0.183724 Aug 27 02:41:57.747302 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 603/1198] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:41:57 2022] GET /compute/v2.1/servers/7f0ec822-1c41-4508-8d13-a48c4c88f133 => generated 1673 bytes in 186 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:41:58.776967 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-11316b5d-34b8-46c0-8d0b-8fbb0ad62d54 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:41:58.777614 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-11316b5d-34b8-46c0-8d0b-8fbb0ad62d54 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] [instance: 7f0ec822-1c41-4508-8d13-a48c4c88f133] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:41:58.779962 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG dbcounter [-] [120863] Writing DB stats nova_api:SELECT=37 {{(pid=120863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 27 02:41:58.785519 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-11316b5d-34b8-46c0-8d0b-8fbb0ad62d54 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:41:58.785995 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-11316b5d-34b8-46c0-8d0b-8fbb0ad62d54 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:41:58.786422 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-11316b5d-34b8-46c0-8d0b-8fbb0ad62d54 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:41:58.787108 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-11316b5d-34b8-46c0-8d0b-8fbb0ad62d54 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:41:58.787552 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-11316b5d-34b8-46c0-8d0b-8fbb0ad62d54 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:41:58.787969 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-11316b5d-34b8-46c0-8d0b-8fbb0ad62d54 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:41:59.002341 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-11316b5d-34b8-46c0-8d0b-8fbb0ad62d54 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee6084fffc544dfe8f1a9a30ae06be69', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a8055f4326f640fc8927f5cfdef39dd3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:41:59.010881 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-11316b5d-34b8-46c0-8d0b-8fbb0ad62d54 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] 10.222.0.52 "GET /compute/v2.1/servers/7f0ec822-1c41-4508-8d13-a48c4c88f133" status: 200 len: 1673 microversion: 2.1 time: 0.237842 Aug 27 02:41:59.011463 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 596/1199] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:41:58 2022] GET /compute/v2.1/servers/7f0ec822-1c41-4508-8d13-a48c4c88f133 => generated 1673 bytes in 239 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:42:00.033336 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-17eede82-6246-4e35-9f90-61dd380b15b1 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Calling method '>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:42:00.038785 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-17eede82-6246-4e35-9f90-61dd380b15b1 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] [instance: 7f0ec822-1c41-4508-8d13-a48c4c88f133] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:42:00.042420 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [req-d4229a38-68e0-45be-9931-01a2abfd9441 req-28dd57a3-1c7e-46b8-a33d-e562f4932825 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "7f0ec822-1c41-4508-8d13-a48c4c88f133", "name": "network-vif-deleted", "tag": "5d767859-f041-4829-989b-8f1b21644308"}]} {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 27 02:42:00.053110 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [req-d4229a38-68e0-45be-9931-01a2abfd9441 req-28dd57a3-1c7e-46b8-a33d-e562f4932825 service nova] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:42:00.053884 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [req-d4229a38-68e0-45be-9931-01a2abfd9441 req-28dd57a3-1c7e-46b8-a33d-e562f4932825 service nova] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:42:00.054595 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [req-d4229a38-68e0-45be-9931-01a2abfd9441 req-28dd57a3-1c7e-46b8-a33d-e562f4932825 service nova] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:42:00.056462 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-17eede82-6246-4e35-9f90-61dd380b15b1 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:42:00.057081 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-17eede82-6246-4e35-9f90-61dd380b15b1 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:42:00.057660 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-17eede82-6246-4e35-9f90-61dd380b15b1 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:42:00.058667 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-17eede82-6246-4e35-9f90-61dd380b15b1 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:42:00.059492 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-17eede82-6246-4e35-9f90-61dd380b15b1 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:42:00.060058 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-17eede82-6246-4e35-9f90-61dd380b15b1 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:42:00.068018 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.compute.server_external_events [req-d4229a38-68e0-45be-9931-01a2abfd9441 req-28dd57a3-1c7e-46b8-a33d-e562f4932825 service nova] Creating event network-vif-deleted:5d767859-f041-4829-989b-8f1b21644308 for instance 7f0ec822-1c41-4508-8d13-a48c4c88f133 on hv-cinder-84630 Aug 27 02:42:00.074477 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [req-d4229a38-68e0-45be-9931-01a2abfd9441 req-28dd57a3-1c7e-46b8-a33d-e562f4932825 service nova] 10.222.0.52 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.035804 Aug 27 02:42:00.075515 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 597/1200] 10.222.0.52 () {64 vars in 1317 bytes} [Sat Aug 27 02:42:00 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 37 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 27 02:42:00.147236 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-17eede82-6246-4e35-9f90-61dd380b15b1 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee6084fffc544dfe8f1a9a30ae06be69', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a8055f4326f640fc8927f5cfdef39dd3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:42:00.157902 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-17eede82-6246-4e35-9f90-61dd380b15b1 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] 10.222.0.52 "GET /compute/v2.1/servers/7f0ec822-1c41-4508-8d13-a48c4c88f133" status: 200 len: 1605 microversion: 2.1 time: 0.128263 Aug 27 02:42:00.158647 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 604/1201] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:42:00 2022] GET /compute/v2.1/servers/7f0ec822-1c41-4508-8d13-a48c4c88f133 => generated 1605 bytes in 130 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:42:01.182205 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-1461a214-55b7-4a90-918d-c92b30b001ab tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:42:01.182782 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-1461a214-55b7-4a90-918d-c92b30b001ab tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] [instance: 7f0ec822-1c41-4508-8d13-a48c4c88f133] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:42:01.189773 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-1461a214-55b7-4a90-918d-c92b30b001ab tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:42:01.190252 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-1461a214-55b7-4a90-918d-c92b30b001ab tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:42:01.190685 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-1461a214-55b7-4a90-918d-c92b30b001ab tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:42:01.191375 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-1461a214-55b7-4a90-918d-c92b30b001ab tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:42:01.191832 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-1461a214-55b7-4a90-918d-c92b30b001ab tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:42:01.192286 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-1461a214-55b7-4a90-918d-c92b30b001ab tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:42:01.206239 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.wsgi [None req-1461a214-55b7-4a90-918d-c92b30b001ab tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] HTTP exception thrown: Instance 7f0ec822-1c41-4508-8d13-a48c4c88f133 could not be found. Aug 27 02:42:01.207871 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-1461a214-55b7-4a90-918d-c92b30b001ab tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Request method failure captured: Aug 27 02:42:01.207871 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: request: GET /compute/v2.1/servers/7f0ec822-1c41-4508-8d13-a48c4c88f133 HTTP/1.1 Aug 27 02:42:01.207871 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: Accept: application/json Aug 27 02:42:01.207871 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: Accept-Encoding: identity Aug 27 02:42:01.207871 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: Connection: close Aug 27 02:42:01.207871 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: Content-Length: 0 Aug 27 02:42:01.207871 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: Content-Type: application/json Aug 27 02:42:01.207871 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: Host: 10.222.0.52 Aug 27 02:42:01.207871 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: Openstack-System-Scope: None Aug 27 02:42:01.207871 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: User-Agent: python-urllib3/1.26.11 Aug 27 02:42:01.207871 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-Auth-Token: gAAAAABjCYLxG4xShIaurn6kMgDB_csdcgHMyuwGl_M1BtZ0jwtkVUCxblTUmXzLHTJtK985z6ro9EscRnvVjjQdqUAN11VYU2wySoyxpoA1-56aKSiuDjDA_a8zJHo26N0yzT8tbkGVw0nbFhpKTK-Au7FefDK9t_CH7NJV_TwOXmOddAnhccQ Aug 27 02:42:01.207871 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-Domain-Id: None Aug 27 02:42:01.207871 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-Domain-Name: None Aug 27 02:42:01.207871 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-Identity-Status: Confirmed Aug 27 02:42:01.207871 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-Is-Admin-Project: True Aug 27 02:42:01.207871 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-Project-Domain-Id: default Aug 27 02:42:01.207871 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-Project-Domain-Name: Default Aug 27 02:42:01.207871 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-Project-Id: a8055f4326f640fc8927f5cfdef39dd3 Aug 27 02:42:01.207871 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-Project-Name: tempest-AttachVolumeTestJSON-1495157572 Aug 27 02:42:01.207871 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-Role: reader,member Aug 27 02:42:01.207871 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-Roles: reader,member Aug 27 02:42:01.209797 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-Service-Catalog: [{"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.52:8080", "publicURL": "http://10.222.0.52:8080/v1/AUTH_a8055f4326f640fc8927f5cfdef39dd3"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52:9696/networking"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/compute/v2/a8055f4326f640fc8927f5cfdef39dd3"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/image"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/volume/v3/a8055f4326f640fc8927f5cfdef39dd3"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/identity"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/volume/v3/a8055f4326f640fc8927f5cfdef39dd3"}]}] Aug 27 02:42:01.209797 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-Tenant: tempest-AttachVolumeTestJSON-1495157572 Aug 27 02:42:01.209797 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-Tenant-Id: a8055f4326f640fc8927f5cfdef39dd3 Aug 27 02:42:01.209797 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-Tenant-Name: tempest-AttachVolumeTestJSON-1495157572 Aug 27 02:42:01.209797 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-User: tempest-AttachVolumeTestJSON-1495157572-project Aug 27 02:42:01.209797 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-User-Domain-Id: default Aug 27 02:42:01.209797 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-User-Domain-Name: Default Aug 27 02:42:01.209797 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-User-Id: ee6084fffc544dfe8f1a9a30ae06be69 Aug 27 02:42:01.209797 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-User-Name: tempest-AttachVolumeTestJSON-1495157572-project Aug 27 02:42:01.209797 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: method: > Aug 27 02:42:01.209797 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: exception: Instance 7f0ec822-1c41-4508-8d13-a48c4c88f133 could not be found. Aug 27 02:42:01.209797 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: action_args: {'id': '7f0ec822-1c41-4508-8d13-a48c4c88f133'} Aug 27 02:42:01.210927 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Aug 27 02:42:01.210927 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 27 02:42:01.210927 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Aug 27 02:42:01.210927 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Aug 27 02:42:01.210927 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Aug 27 02:42:01.210927 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Aug 27 02:42:01.210927 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Aug 27 02:42:01.210927 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Aug 27 02:42:01.210927 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Aug 27 02:42:01.210927 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Aug 27 02:42:01.210927 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 7f0ec822-1c41-4508-8d13-a48c4c88f133 could not be found. Aug 27 02:42:01.210927 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi Aug 27 02:42:01.210927 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 27 02:42:01.210927 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi Aug 27 02:42:01.210927 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 27 02:42:01.210927 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 27 02:42:01.210927 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 27 02:42:01.210927 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Aug 27 02:42:01.210927 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Aug 27 02:42:01.210927 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Aug 27 02:42:01.212568 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Aug 27 02:42:01.212568 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Aug 27 02:42:01.212568 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi instance = self._get_server( Aug 27 02:42:01.212568 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Aug 27 02:42:01.212568 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Aug 27 02:42:01.212568 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Aug 27 02:42:01.212568 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Aug 27 02:42:01.212568 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 7f0ec822-1c41-4508-8d13-a48c4c88f133 could not be found. Aug 27 02:42:01.212568 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi Aug 27 02:42:01.212568 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 27 02:42:01.212568 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi Aug 27 02:42:01.212568 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 27 02:42:01.212568 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 27 02:42:01.212568 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 27 02:42:01.212568 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Aug 27 02:42:01.212568 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Aug 27 02:42:01.212568 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 7f0ec822-1c41-4508-8d13-a48c4c88f133 could not be found. Aug 27 02:42:01.212568 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi Aug 27 02:42:01.214375 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-1461a214-55b7-4a90-918d-c92b30b001ab tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Returning 404 to user: Instance 7f0ec822-1c41-4508-8d13-a48c4c88f133 could not be found. {{(pid=120863) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Aug 27 02:42:01.214375 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-1461a214-55b7-4a90-918d-c92b30b001ab tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] 10.222.0.52 "GET /compute/v2.1/servers/7f0ec822-1c41-4508-8d13-a48c4c88f133" status: 404 len: 111 microversion: 2.1 time: 0.032249 Aug 27 02:42:01.214375 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 598/1202] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:42:01 2022] GET /compute/v2.1/servers/7f0ec822-1c41-4508-8d13-a48c4c88f133 => generated 111 bytes in 34 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Aug 27 02:42:01.218264 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-4dafa78c-7dce-44f0-a03a-8a826bb05ef1 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Calling method '.version_select at 0x7f6b1e8924c0>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:42:01.242399 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-4dafa78c-7dce-44f0-a03a-8a826bb05ef1 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] 10.222.0.52 "DELETE /compute/v2.1/os-keypairs/tempest-keypair-630444685" status: 202 len: 0 microversion: 2.1 time: 0.027595 Aug 27 02:42:01.243227 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 605/1203] 10.222.0.52 () {58 vars in 1218 bytes} [Sat Aug 27 02:42:01 2022] DELETE /compute/v2.1/os-keypairs/tempest-keypair-630444685 => generated 0 bytes in 29 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Aug 27 02:42:04.218927 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-09dc3134-d660-4603-b206-cf946f7001ab tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:42:04.219445 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-09dc3134-d660-4603-b206-cf946f7001ab tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] [instance: 7f0ec822-1c41-4508-8d13-a48c4c88f133] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:42:04.226692 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-09dc3134-d660-4603-b206-cf946f7001ab tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:42:04.227174 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-09dc3134-d660-4603-b206-cf946f7001ab tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:42:04.227612 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-09dc3134-d660-4603-b206-cf946f7001ab tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:42:04.228352 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-09dc3134-d660-4603-b206-cf946f7001ab tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:42:04.228764 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-09dc3134-d660-4603-b206-cf946f7001ab tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:42:04.229224 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-09dc3134-d660-4603-b206-cf946f7001ab tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:42:04.243310 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.wsgi [None req-09dc3134-d660-4603-b206-cf946f7001ab tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] HTTP exception thrown: Instance 7f0ec822-1c41-4508-8d13-a48c4c88f133 could not be found. Aug 27 02:42:04.244852 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-09dc3134-d660-4603-b206-cf946f7001ab tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Request method failure captured: Aug 27 02:42:04.244852 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: request: DELETE /compute/v2.1/servers/7f0ec822-1c41-4508-8d13-a48c4c88f133 HTTP/1.1 Aug 27 02:42:04.244852 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: Accept: application/json Aug 27 02:42:04.244852 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: Accept-Encoding: identity Aug 27 02:42:04.244852 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: Connection: close Aug 27 02:42:04.244852 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: Content-Length: 0 Aug 27 02:42:04.244852 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: Content-Type: application/json Aug 27 02:42:04.244852 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: Host: 10.222.0.52 Aug 27 02:42:04.244852 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: Openstack-System-Scope: None Aug 27 02:42:04.244852 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: User-Agent: python-urllib3/1.26.11 Aug 27 02:42:04.244852 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-Auth-Token: gAAAAABjCYLxG4xShIaurn6kMgDB_csdcgHMyuwGl_M1BtZ0jwtkVUCxblTUmXzLHTJtK985z6ro9EscRnvVjjQdqUAN11VYU2wySoyxpoA1-56aKSiuDjDA_a8zJHo26N0yzT8tbkGVw0nbFhpKTK-Au7FefDK9t_CH7NJV_TwOXmOddAnhccQ Aug 27 02:42:04.244852 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-Domain-Id: None Aug 27 02:42:04.244852 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-Domain-Name: None Aug 27 02:42:04.244852 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-Identity-Status: Confirmed Aug 27 02:42:04.244852 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-Is-Admin-Project: True Aug 27 02:42:04.244852 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-Project-Domain-Id: default Aug 27 02:42:04.244852 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-Project-Domain-Name: Default Aug 27 02:42:04.244852 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-Project-Id: a8055f4326f640fc8927f5cfdef39dd3 Aug 27 02:42:04.244852 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-Project-Name: tempest-AttachVolumeTestJSON-1495157572 Aug 27 02:42:04.244852 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-Role: reader,member Aug 27 02:42:04.244852 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-Roles: reader,member Aug 27 02:42:04.246482 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-Service-Catalog: [{"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.52:8080", "publicURL": "http://10.222.0.52:8080/v1/AUTH_a8055f4326f640fc8927f5cfdef39dd3"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52:9696/networking"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/compute/v2/a8055f4326f640fc8927f5cfdef39dd3"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/image"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/volume/v3/a8055f4326f640fc8927f5cfdef39dd3"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/identity"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/volume/v3/a8055f4326f640fc8927f5cfdef39dd3"}]}] Aug 27 02:42:04.246482 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-Tenant: tempest-AttachVolumeTestJSON-1495157572 Aug 27 02:42:04.246482 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-Tenant-Id: a8055f4326f640fc8927f5cfdef39dd3 Aug 27 02:42:04.246482 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-Tenant-Name: tempest-AttachVolumeTestJSON-1495157572 Aug 27 02:42:04.246482 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-User: tempest-AttachVolumeTestJSON-1495157572-project Aug 27 02:42:04.246482 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-User-Domain-Id: default Aug 27 02:42:04.246482 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-User-Domain-Name: Default Aug 27 02:42:04.246482 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-User-Id: ee6084fffc544dfe8f1a9a30ae06be69 Aug 27 02:42:04.246482 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-User-Name: tempest-AttachVolumeTestJSON-1495157572-project Aug 27 02:42:04.246482 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: method: > Aug 27 02:42:04.246482 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: exception: Instance 7f0ec822-1c41-4508-8d13-a48c4c88f133 could not be found. Aug 27 02:42:04.246482 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: action_args: {'id': '7f0ec822-1c41-4508-8d13-a48c4c88f133'} Aug 27 02:42:04.247300 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Aug 27 02:42:04.247300 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 27 02:42:04.247300 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Aug 27 02:42:04.247300 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Aug 27 02:42:04.247300 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Aug 27 02:42:04.247300 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Aug 27 02:42:04.247300 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Aug 27 02:42:04.247300 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Aug 27 02:42:04.247300 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Aug 27 02:42:04.247300 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Aug 27 02:42:04.247300 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 7f0ec822-1c41-4508-8d13-a48c4c88f133 could not be found. Aug 27 02:42:04.247300 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi Aug 27 02:42:04.247300 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 27 02:42:04.247300 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi Aug 27 02:42:04.247300 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 27 02:42:04.247300 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 27 02:42:04.247300 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 27 02:42:04.247300 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Aug 27 02:42:04.247300 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Aug 27 02:42:04.247300 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Aug 27 02:42:04.248810 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Aug 27 02:42:04.248810 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Aug 27 02:42:04.248810 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Aug 27 02:42:04.248810 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Aug 27 02:42:04.248810 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Aug 27 02:42:04.248810 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Aug 27 02:42:04.248810 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Aug 27 02:42:04.248810 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Aug 27 02:42:04.248810 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Aug 27 02:42:04.248810 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 7f0ec822-1c41-4508-8d13-a48c4c88f133 could not be found. Aug 27 02:42:04.248810 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi Aug 27 02:42:04.248810 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 27 02:42:04.248810 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi Aug 27 02:42:04.248810 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 27 02:42:04.248810 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 27 02:42:04.248810 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 27 02:42:04.248810 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Aug 27 02:42:04.248810 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Aug 27 02:42:04.248810 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 7f0ec822-1c41-4508-8d13-a48c4c88f133 could not be found. Aug 27 02:42:04.248810 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi Aug 27 02:42:04.250422 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-09dc3134-d660-4603-b206-cf946f7001ab tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Returning 404 to user: Instance 7f0ec822-1c41-4508-8d13-a48c4c88f133 could not be found. {{(pid=120863) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Aug 27 02:42:04.250422 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-09dc3134-d660-4603-b206-cf946f7001ab tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] 10.222.0.52 "DELETE /compute/v2.1/servers/7f0ec822-1c41-4508-8d13-a48c4c88f133" status: 404 len: 111 microversion: 2.1 time: 0.031232 Aug 27 02:42:04.250422 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 599/1204] 10.222.0.52 () {58 vars in 1239 bytes} [Sat Aug 27 02:42:04 2022] DELETE /compute/v2.1/servers/7f0ec822-1c41-4508-8d13-a48c4c88f133 => generated 111 bytes in 32 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Aug 27 02:42:04.255473 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-e03f4f3b-aff0-43ca-924a-518be75c2827 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Calling method '>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:42:04.256082 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-e03f4f3b-aff0-43ca-924a-518be75c2827 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] [instance: 7f0ec822-1c41-4508-8d13-a48c4c88f133] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:42:04.264070 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-e03f4f3b-aff0-43ca-924a-518be75c2827 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:42:04.264558 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-e03f4f3b-aff0-43ca-924a-518be75c2827 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:42:04.265016 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-e03f4f3b-aff0-43ca-924a-518be75c2827 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:42:04.265738 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-e03f4f3b-aff0-43ca-924a-518be75c2827 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:42:04.266212 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-e03f4f3b-aff0-43ca-924a-518be75c2827 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:42:04.266589 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-e03f4f3b-aff0-43ca-924a-518be75c2827 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:42:04.281072 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.wsgi [None req-e03f4f3b-aff0-43ca-924a-518be75c2827 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] HTTP exception thrown: Instance 7f0ec822-1c41-4508-8d13-a48c4c88f133 could not be found. Aug 27 02:42:04.282765 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-e03f4f3b-aff0-43ca-924a-518be75c2827 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Request method failure captured: Aug 27 02:42:04.282765 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: request: GET /compute/v2.1/servers/7f0ec822-1c41-4508-8d13-a48c4c88f133 HTTP/1.1 Aug 27 02:42:04.282765 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: Accept: application/json Aug 27 02:42:04.282765 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: Accept-Encoding: identity Aug 27 02:42:04.282765 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: Connection: close Aug 27 02:42:04.282765 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: Content-Length: 0 Aug 27 02:42:04.282765 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: Content-Type: application/json Aug 27 02:42:04.282765 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: Host: 10.222.0.52 Aug 27 02:42:04.282765 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: Openstack-System-Scope: None Aug 27 02:42:04.282765 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: User-Agent: python-urllib3/1.26.11 Aug 27 02:42:04.282765 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-Auth-Token: gAAAAABjCYLxG4xShIaurn6kMgDB_csdcgHMyuwGl_M1BtZ0jwtkVUCxblTUmXzLHTJtK985z6ro9EscRnvVjjQdqUAN11VYU2wySoyxpoA1-56aKSiuDjDA_a8zJHo26N0yzT8tbkGVw0nbFhpKTK-Au7FefDK9t_CH7NJV_TwOXmOddAnhccQ Aug 27 02:42:04.282765 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-Domain-Id: None Aug 27 02:42:04.282765 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-Domain-Name: None Aug 27 02:42:04.282765 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-Identity-Status: Confirmed Aug 27 02:42:04.282765 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-Is-Admin-Project: True Aug 27 02:42:04.282765 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-Project-Domain-Id: default Aug 27 02:42:04.282765 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-Project-Domain-Name: Default Aug 27 02:42:04.282765 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-Project-Id: a8055f4326f640fc8927f5cfdef39dd3 Aug 27 02:42:04.282765 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-Project-Name: tempest-AttachVolumeTestJSON-1495157572 Aug 27 02:42:04.282765 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-Role: reader,member Aug 27 02:42:04.282765 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-Roles: reader,member Aug 27 02:42:04.284434 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-Service-Catalog: [{"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.52:8080", "publicURL": "http://10.222.0.52:8080/v1/AUTH_a8055f4326f640fc8927f5cfdef39dd3"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52:9696/networking"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/compute/v2/a8055f4326f640fc8927f5cfdef39dd3"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/image"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/volume/v3/a8055f4326f640fc8927f5cfdef39dd3"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/identity"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/volume/v3/a8055f4326f640fc8927f5cfdef39dd3"}]}] Aug 27 02:42:04.284434 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-Tenant: tempest-AttachVolumeTestJSON-1495157572 Aug 27 02:42:04.284434 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-Tenant-Id: a8055f4326f640fc8927f5cfdef39dd3 Aug 27 02:42:04.284434 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-Tenant-Name: tempest-AttachVolumeTestJSON-1495157572 Aug 27 02:42:04.284434 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-User: tempest-AttachVolumeTestJSON-1495157572-project Aug 27 02:42:04.284434 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-User-Domain-Id: default Aug 27 02:42:04.284434 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-User-Domain-Name: Default Aug 27 02:42:04.284434 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-User-Id: ee6084fffc544dfe8f1a9a30ae06be69 Aug 27 02:42:04.284434 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-User-Name: tempest-AttachVolumeTestJSON-1495157572-project Aug 27 02:42:04.284434 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: method: > Aug 27 02:42:04.284434 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: exception: Instance 7f0ec822-1c41-4508-8d13-a48c4c88f133 could not be found. Aug 27 02:42:04.284434 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: action_args: {'id': '7f0ec822-1c41-4508-8d13-a48c4c88f133'} Aug 27 02:42:04.285560 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Aug 27 02:42:04.285560 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 27 02:42:04.285560 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Aug 27 02:42:04.285560 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Aug 27 02:42:04.285560 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Aug 27 02:42:04.285560 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Aug 27 02:42:04.285560 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Aug 27 02:42:04.285560 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Aug 27 02:42:04.285560 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Aug 27 02:42:04.285560 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Aug 27 02:42:04.285560 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 7f0ec822-1c41-4508-8d13-a48c4c88f133 could not be found. Aug 27 02:42:04.285560 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi Aug 27 02:42:04.285560 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 27 02:42:04.285560 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi Aug 27 02:42:04.285560 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 27 02:42:04.285560 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 27 02:42:04.285560 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 27 02:42:04.285560 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Aug 27 02:42:04.285560 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Aug 27 02:42:04.285560 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Aug 27 02:42:04.287173 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Aug 27 02:42:04.287173 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Aug 27 02:42:04.287173 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi instance = self._get_server( Aug 27 02:42:04.287173 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Aug 27 02:42:04.287173 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Aug 27 02:42:04.287173 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Aug 27 02:42:04.287173 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Aug 27 02:42:04.287173 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 7f0ec822-1c41-4508-8d13-a48c4c88f133 could not be found. Aug 27 02:42:04.287173 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi Aug 27 02:42:04.287173 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 27 02:42:04.287173 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi Aug 27 02:42:04.287173 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 27 02:42:04.287173 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 27 02:42:04.287173 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 27 02:42:04.287173 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Aug 27 02:42:04.287173 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Aug 27 02:42:04.287173 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 7f0ec822-1c41-4508-8d13-a48c4c88f133 could not be found. Aug 27 02:42:04.287173 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi Aug 27 02:42:04.289402 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-e03f4f3b-aff0-43ca-924a-518be75c2827 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Returning 404 to user: Instance 7f0ec822-1c41-4508-8d13-a48c4c88f133 could not be found. {{(pid=120864) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Aug 27 02:42:04.289402 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-e03f4f3b-aff0-43ca-924a-518be75c2827 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] 10.222.0.52 "GET /compute/v2.1/servers/7f0ec822-1c41-4508-8d13-a48c4c88f133" status: 404 len: 111 microversion: 2.1 time: 0.033160 Aug 27 02:42:04.289402 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 606/1205] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:42:04 2022] GET /compute/v2.1/servers/7f0ec822-1c41-4508-8d13-a48c4c88f133 => generated 111 bytes in 34 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Aug 27 02:42:05.461839 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-2f9642e3-dcd3-481a-9c29-196c4c65eba1 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:42:05.462360 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-2f9642e3-dcd3-481a-9c29-196c4c65eba1 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] [instance: f5bad70c-350c-4230-a862-626dd6805e4c] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:42:05.472854 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-2f9642e3-dcd3-481a-9c29-196c4c65eba1 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:42:05.473552 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-2f9642e3-dcd3-481a-9c29-196c4c65eba1 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:42:05.474155 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-2f9642e3-dcd3-481a-9c29-196c4c65eba1 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:42:05.474867 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-2f9642e3-dcd3-481a-9c29-196c4c65eba1 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:42:05.475327 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-2f9642e3-dcd3-481a-9c29-196c4c65eba1 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:42:05.475844 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-2f9642e3-dcd3-481a-9c29-196c4c65eba1 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:42:05.490297 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.wsgi [None req-2f9642e3-dcd3-481a-9c29-196c4c65eba1 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] HTTP exception thrown: Instance f5bad70c-350c-4230-a862-626dd6805e4c could not be found. Aug 27 02:42:05.492251 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-2f9642e3-dcd3-481a-9c29-196c4c65eba1 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Request method failure captured: Aug 27 02:42:05.492251 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: request: DELETE /compute/v2.1/servers/f5bad70c-350c-4230-a862-626dd6805e4c HTTP/1.1 Aug 27 02:42:05.492251 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: Accept: application/json Aug 27 02:42:05.492251 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: Accept-Encoding: identity Aug 27 02:42:05.492251 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: Connection: close Aug 27 02:42:05.492251 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: Content-Length: 0 Aug 27 02:42:05.492251 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: Content-Type: application/json Aug 27 02:42:05.492251 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: Host: 10.222.0.52 Aug 27 02:42:05.492251 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: Openstack-System-Scope: None Aug 27 02:42:05.492251 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: User-Agent: python-urllib3/1.26.11 Aug 27 02:42:05.492251 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-Auth-Token: gAAAAABjCYLxG4xShIaurn6kMgDB_csdcgHMyuwGl_M1BtZ0jwtkVUCxblTUmXzLHTJtK985z6ro9EscRnvVjjQdqUAN11VYU2wySoyxpoA1-56aKSiuDjDA_a8zJHo26N0yzT8tbkGVw0nbFhpKTK-Au7FefDK9t_CH7NJV_TwOXmOddAnhccQ Aug 27 02:42:05.492251 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-Domain-Id: None Aug 27 02:42:05.492251 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-Domain-Name: None Aug 27 02:42:05.492251 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-Identity-Status: Confirmed Aug 27 02:42:05.492251 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-Is-Admin-Project: True Aug 27 02:42:05.492251 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-Project-Domain-Id: default Aug 27 02:42:05.492251 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-Project-Domain-Name: Default Aug 27 02:42:05.492251 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-Project-Id: a8055f4326f640fc8927f5cfdef39dd3 Aug 27 02:42:05.492251 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-Project-Name: tempest-AttachVolumeTestJSON-1495157572 Aug 27 02:42:05.492251 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-Role: reader,member Aug 27 02:42:05.492251 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-Roles: reader,member Aug 27 02:42:05.493993 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-Service-Catalog: [{"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.52:8080", "publicURL": "http://10.222.0.52:8080/v1/AUTH_a8055f4326f640fc8927f5cfdef39dd3"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52:9696/networking"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/compute/v2/a8055f4326f640fc8927f5cfdef39dd3"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/image"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/volume/v3/a8055f4326f640fc8927f5cfdef39dd3"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/identity"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/volume/v3/a8055f4326f640fc8927f5cfdef39dd3"}]}] Aug 27 02:42:05.493993 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-Tenant: tempest-AttachVolumeTestJSON-1495157572 Aug 27 02:42:05.493993 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-Tenant-Id: a8055f4326f640fc8927f5cfdef39dd3 Aug 27 02:42:05.493993 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-Tenant-Name: tempest-AttachVolumeTestJSON-1495157572 Aug 27 02:42:05.493993 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-User: tempest-AttachVolumeTestJSON-1495157572-project Aug 27 02:42:05.493993 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-User-Domain-Id: default Aug 27 02:42:05.493993 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-User-Domain-Name: Default Aug 27 02:42:05.493993 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-User-Id: ee6084fffc544dfe8f1a9a30ae06be69 Aug 27 02:42:05.493993 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-User-Name: tempest-AttachVolumeTestJSON-1495157572-project Aug 27 02:42:05.493993 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: method: > Aug 27 02:42:05.493993 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: exception: Instance f5bad70c-350c-4230-a862-626dd6805e4c could not be found. Aug 27 02:42:05.493993 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: action_args: {'id': 'f5bad70c-350c-4230-a862-626dd6805e4c'} Aug 27 02:42:05.494944 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Aug 27 02:42:05.494944 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 27 02:42:05.494944 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Aug 27 02:42:05.494944 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Aug 27 02:42:05.494944 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Aug 27 02:42:05.494944 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Aug 27 02:42:05.494944 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Aug 27 02:42:05.494944 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Aug 27 02:42:05.494944 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Aug 27 02:42:05.494944 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Aug 27 02:42:05.494944 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance f5bad70c-350c-4230-a862-626dd6805e4c could not be found. Aug 27 02:42:05.494944 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi Aug 27 02:42:05.494944 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 27 02:42:05.494944 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi Aug 27 02:42:05.494944 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 27 02:42:05.494944 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 27 02:42:05.494944 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 27 02:42:05.494944 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Aug 27 02:42:05.494944 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Aug 27 02:42:05.494944 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Aug 27 02:42:05.496611 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Aug 27 02:42:05.496611 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Aug 27 02:42:05.496611 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Aug 27 02:42:05.496611 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Aug 27 02:42:05.496611 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Aug 27 02:42:05.496611 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Aug 27 02:42:05.496611 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Aug 27 02:42:05.496611 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Aug 27 02:42:05.496611 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Aug 27 02:42:05.496611 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance f5bad70c-350c-4230-a862-626dd6805e4c could not be found. Aug 27 02:42:05.496611 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi Aug 27 02:42:05.496611 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 27 02:42:05.496611 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi Aug 27 02:42:05.496611 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 27 02:42:05.496611 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 27 02:42:05.496611 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 27 02:42:05.496611 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Aug 27 02:42:05.496611 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Aug 27 02:42:05.496611 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance f5bad70c-350c-4230-a862-626dd6805e4c could not be found. Aug 27 02:42:05.496611 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi Aug 27 02:42:05.498386 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-2f9642e3-dcd3-481a-9c29-196c4c65eba1 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Returning 404 to user: Instance f5bad70c-350c-4230-a862-626dd6805e4c could not be found. {{(pid=120863) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Aug 27 02:42:05.498386 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-2f9642e3-dcd3-481a-9c29-196c4c65eba1 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] 10.222.0.52 "DELETE /compute/v2.1/servers/f5bad70c-350c-4230-a862-626dd6805e4c" status: 404 len: 111 microversion: 2.1 time: 0.038002 Aug 27 02:42:05.498386 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 600/1206] 10.222.0.52 () {58 vars in 1239 bytes} [Sat Aug 27 02:42:05 2022] DELETE /compute/v2.1/servers/f5bad70c-350c-4230-a862-626dd6805e4c => generated 111 bytes in 39 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Aug 27 02:42:05.502865 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-470dfbbc-f520-4901-982d-aa9f4ffae421 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Calling method '>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:42:05.503496 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-470dfbbc-f520-4901-982d-aa9f4ffae421 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] [instance: f5bad70c-350c-4230-a862-626dd6805e4c] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:42:05.510400 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-470dfbbc-f520-4901-982d-aa9f4ffae421 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:42:05.510876 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-470dfbbc-f520-4901-982d-aa9f4ffae421 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:42:05.511306 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-470dfbbc-f520-4901-982d-aa9f4ffae421 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:42:05.511996 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-470dfbbc-f520-4901-982d-aa9f4ffae421 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:42:05.512442 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-470dfbbc-f520-4901-982d-aa9f4ffae421 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:42:05.512861 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-470dfbbc-f520-4901-982d-aa9f4ffae421 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:42:05.526657 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.wsgi [None req-470dfbbc-f520-4901-982d-aa9f4ffae421 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] HTTP exception thrown: Instance f5bad70c-350c-4230-a862-626dd6805e4c could not be found. Aug 27 02:42:05.528170 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-470dfbbc-f520-4901-982d-aa9f4ffae421 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Request method failure captured: Aug 27 02:42:05.528170 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: request: GET /compute/v2.1/servers/f5bad70c-350c-4230-a862-626dd6805e4c HTTP/1.1 Aug 27 02:42:05.528170 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: Accept: application/json Aug 27 02:42:05.528170 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: Accept-Encoding: identity Aug 27 02:42:05.528170 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: Connection: close Aug 27 02:42:05.528170 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: Content-Length: 0 Aug 27 02:42:05.528170 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: Content-Type: application/json Aug 27 02:42:05.528170 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: Host: 10.222.0.52 Aug 27 02:42:05.528170 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: Openstack-System-Scope: None Aug 27 02:42:05.528170 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: User-Agent: python-urllib3/1.26.11 Aug 27 02:42:05.528170 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-Auth-Token: gAAAAABjCYLxG4xShIaurn6kMgDB_csdcgHMyuwGl_M1BtZ0jwtkVUCxblTUmXzLHTJtK985z6ro9EscRnvVjjQdqUAN11VYU2wySoyxpoA1-56aKSiuDjDA_a8zJHo26N0yzT8tbkGVw0nbFhpKTK-Au7FefDK9t_CH7NJV_TwOXmOddAnhccQ Aug 27 02:42:05.528170 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-Domain-Id: None Aug 27 02:42:05.528170 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-Domain-Name: None Aug 27 02:42:05.528170 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-Identity-Status: Confirmed Aug 27 02:42:05.528170 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-Is-Admin-Project: True Aug 27 02:42:05.528170 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-Project-Domain-Id: default Aug 27 02:42:05.528170 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-Project-Domain-Name: Default Aug 27 02:42:05.528170 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-Project-Id: a8055f4326f640fc8927f5cfdef39dd3 Aug 27 02:42:05.528170 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-Project-Name: tempest-AttachVolumeTestJSON-1495157572 Aug 27 02:42:05.528170 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-Role: reader,member Aug 27 02:42:05.528170 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-Roles: reader,member Aug 27 02:42:05.529825 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-Service-Catalog: [{"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.52:8080", "publicURL": "http://10.222.0.52:8080/v1/AUTH_a8055f4326f640fc8927f5cfdef39dd3"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52:9696/networking"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/compute/v2/a8055f4326f640fc8927f5cfdef39dd3"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/image"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/volume/v3/a8055f4326f640fc8927f5cfdef39dd3"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/identity"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/volume/v3/a8055f4326f640fc8927f5cfdef39dd3"}]}] Aug 27 02:42:05.529825 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-Tenant: tempest-AttachVolumeTestJSON-1495157572 Aug 27 02:42:05.529825 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-Tenant-Id: a8055f4326f640fc8927f5cfdef39dd3 Aug 27 02:42:05.529825 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-Tenant-Name: tempest-AttachVolumeTestJSON-1495157572 Aug 27 02:42:05.529825 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-User: tempest-AttachVolumeTestJSON-1495157572-project Aug 27 02:42:05.529825 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-User-Domain-Id: default Aug 27 02:42:05.529825 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-User-Domain-Name: Default Aug 27 02:42:05.529825 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-User-Id: ee6084fffc544dfe8f1a9a30ae06be69 Aug 27 02:42:05.529825 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-User-Name: tempest-AttachVolumeTestJSON-1495157572-project Aug 27 02:42:05.529825 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: method: > Aug 27 02:42:05.529825 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: exception: Instance f5bad70c-350c-4230-a862-626dd6805e4c could not be found. Aug 27 02:42:05.529825 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: action_args: {'id': 'f5bad70c-350c-4230-a862-626dd6805e4c'} Aug 27 02:42:05.531057 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Aug 27 02:42:05.531057 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 27 02:42:05.531057 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Aug 27 02:42:05.531057 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Aug 27 02:42:05.531057 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Aug 27 02:42:05.531057 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Aug 27 02:42:05.531057 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Aug 27 02:42:05.531057 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Aug 27 02:42:05.531057 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Aug 27 02:42:05.531057 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Aug 27 02:42:05.531057 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance f5bad70c-350c-4230-a862-626dd6805e4c could not be found. Aug 27 02:42:05.531057 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi Aug 27 02:42:05.531057 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 27 02:42:05.531057 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi Aug 27 02:42:05.531057 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 27 02:42:05.531057 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 27 02:42:05.531057 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 27 02:42:05.531057 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Aug 27 02:42:05.531057 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Aug 27 02:42:05.531057 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Aug 27 02:42:05.532775 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Aug 27 02:42:05.532775 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Aug 27 02:42:05.532775 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi instance = self._get_server( Aug 27 02:42:05.532775 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Aug 27 02:42:05.532775 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Aug 27 02:42:05.532775 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Aug 27 02:42:05.532775 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Aug 27 02:42:05.532775 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance f5bad70c-350c-4230-a862-626dd6805e4c could not be found. Aug 27 02:42:05.532775 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi Aug 27 02:42:05.532775 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 27 02:42:05.532775 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi Aug 27 02:42:05.532775 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 27 02:42:05.532775 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 27 02:42:05.532775 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 27 02:42:05.532775 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Aug 27 02:42:05.532775 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Aug 27 02:42:05.532775 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance f5bad70c-350c-4230-a862-626dd6805e4c could not be found. Aug 27 02:42:05.532775 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi Aug 27 02:42:05.534406 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-470dfbbc-f520-4901-982d-aa9f4ffae421 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] Returning 404 to user: Instance f5bad70c-350c-4230-a862-626dd6805e4c could not be found. {{(pid=120864) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Aug 27 02:42:05.534406 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-470dfbbc-f520-4901-982d-aa9f4ffae421 tempest-AttachVolumeTestJSON-1495157572 tempest-AttachVolumeTestJSON-1495157572-project] 10.222.0.52 "GET /compute/v2.1/servers/f5bad70c-350c-4230-a862-626dd6805e4c" status: 404 len: 111 microversion: 2.1 time: 0.031066 Aug 27 02:42:05.534406 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 607/1207] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:42:05 2022] GET /compute/v2.1/servers/f5bad70c-350c-4230-a862-626dd6805e4c => generated 111 bytes in 32 msecs (HTTP/1.1 404) 9 headers in 379 bytes (2 switches on core 0) Aug 27 02:42:56.819159 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-97df6e33-44e0-415e-8498-6f85bcb9defe tempest-TestMinimumBasicScenario-1447275400 tempest-TestMinimumBasicScenario-1447275400-project] Action: 'create', calling method: .version_select at 0x7f6b1b1e1ca0>, body: {"keypair": {"name": "tempest-TestMinimumBasicScenario-970568034", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBCegDtsknGZmUhHWv0ijfzsBdwteTXcsanoLzmiV0a8KYCSatqtwVPW+IeBM5hxQU1wcr5tn5C+atUi8KOHdgV04eMl06ymZ8PQa5jQNInycdgtB92bvjbDbylQQeU9/tA=="}} {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 27 02:42:56.824967 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG dbcounter [-] [120863] Writing DB stats nova_api:SELECT=9 {{(pid=120863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 27 02:42:56.825692 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG dbcounter [-] [120863] Writing DB stats nova_cell1:SELECT=24 {{(pid=120863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 27 02:42:56.838587 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.quota [None req-97df6e33-44e0-415e-8498-6f85bcb9defe tempest-TestMinimumBasicScenario-1447275400 tempest-TestMinimumBasicScenario-1447275400-project] Getting quotas for project 7dd5f20c4a714ee3b555d4aef8b87730. Resources: {'key_pairs'} {{(pid=120863) _get_quotas /opt/stack/nova/nova/quota.py:393}} Aug 27 02:42:56.842898 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.quota [None req-97df6e33-44e0-415e-8498-6f85bcb9defe tempest-TestMinimumBasicScenario-1447275400 tempest-TestMinimumBasicScenario-1447275400-project] Getting quotas for user ba73716667184588b522bc7856404be8 and project 7dd5f20c4a714ee3b555d4aef8b87730. Resources: {'key_pairs'} {{(pid=120863) _get_quotas /opt/stack/nova/nova/quota.py:383}} Aug 27 02:42:56.867964 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-97df6e33-44e0-415e-8498-6f85bcb9defe tempest-TestMinimumBasicScenario-1447275400 tempest-TestMinimumBasicScenario-1447275400-project] 10.222.0.52 "POST /compute/v2.1/os-keypairs" status: 200 len: 402 microversion: 2.1 time: 0.052921 Aug 27 02:42:56.868552 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 601/1208] 10.222.0.52 () {60 vars in 1159 bytes} [Sat Aug 27 02:42:56 2022] POST /compute/v2.1/os-keypairs => generated 402 bytes in 54 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 27 02:42:56.931538 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-02b3ff3d-a872-4015-bc59-04e7961d2bdb tempest-TestMinimumBasicScenario-1447275400 tempest-TestMinimumBasicScenario-1447275400-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-TestMinimumBasicScenario-server-2078435423", "imageRef": "8e8edbe7-973c-4c06-8f7c-349dcb775509", "flavorRef": "452", "key_name": "tempest-TestMinimumBasicScenario-970568034", "networks": [{"uuid": "5e0006d3-60f4-4782-a7c8-edf3b2df76c4"}]}} {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 27 02:42:56.944517 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG dbcounter [-] [120864] Writing DB stats nova_api:SELECT=10,nova_api:DELETE=1 {{(pid=120864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 27 02:42:56.945169 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG dbcounter [-] [120864] Writing DB stats nova_cell1:SELECT=24,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=120864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 27 02:42:56.996787 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.network.neutron [None req-02b3ff3d-a872-4015-bc59-04e7961d2bdb tempest-TestMinimumBasicScenario-1447275400 tempest-TestMinimumBasicScenario-1447275400-project] validate_networks() for [('5e0006d3-60f4-4782-a7c8-edf3b2df76c4', None, None, None, None, None)] {{(pid=120864) validate_networks /opt/stack/nova/nova/network/neutron.py:2626}} Aug 27 02:42:57.424854 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.virt.hardware [None req-02b3ff3d-a872-4015-bc59-04e7961d2bdb tempest-TestMinimumBasicScenario-1447275400 tempest-TestMinimumBasicScenario-1447275400-project] Flavor limits 0:0:0 {{(pid=120864) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Aug 27 02:42:57.425278 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.virt.hardware [None req-02b3ff3d-a872-4015-bc59-04e7961d2bdb tempest-TestMinimumBasicScenario-1447275400 tempest-TestMinimumBasicScenario-1447275400-project] Image limits 0:0:0 {{(pid=120864) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Aug 27 02:42:57.426166 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.virt.hardware [None req-02b3ff3d-a872-4015-bc59-04e7961d2bdb tempest-TestMinimumBasicScenario-1447275400 tempest-TestMinimumBasicScenario-1447275400-project] Flavor pref 0:0:0 {{(pid=120864) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Aug 27 02:42:57.426699 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.virt.hardware [None req-02b3ff3d-a872-4015-bc59-04e7961d2bdb tempest-TestMinimumBasicScenario-1447275400 tempest-TestMinimumBasicScenario-1447275400-project] Image pref 0:0:0 {{(pid=120864) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Aug 27 02:42:57.427117 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.virt.hardware [None req-02b3ff3d-a872-4015-bc59-04e7961d2bdb tempest-TestMinimumBasicScenario-1447275400 tempest-TestMinimumBasicScenario-1447275400-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=120864) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Aug 27 02:42:57.436960 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-02b3ff3d-a872-4015-bc59-04e7961d2bdb tempest-TestMinimumBasicScenario-1447275400 tempest-TestMinimumBasicScenario-1447275400-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:42:57.437466 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-02b3ff3d-a872-4015-bc59-04e7961d2bdb tempest-TestMinimumBasicScenario-1447275400 tempest-TestMinimumBasicScenario-1447275400-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:42:57.437898 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-02b3ff3d-a872-4015-bc59-04e7961d2bdb tempest-TestMinimumBasicScenario-1447275400 tempest-TestMinimumBasicScenario-1447275400-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:42:57.438614 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-02b3ff3d-a872-4015-bc59-04e7961d2bdb tempest-TestMinimumBasicScenario-1447275400 tempest-TestMinimumBasicScenario-1447275400-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:42:57.439056 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-02b3ff3d-a872-4015-bc59-04e7961d2bdb tempest-TestMinimumBasicScenario-1447275400 tempest-TestMinimumBasicScenario-1447275400-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:42:57.439473 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-02b3ff3d-a872-4015-bc59-04e7961d2bdb tempest-TestMinimumBasicScenario-1447275400 tempest-TestMinimumBasicScenario-1447275400-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:42:57.456401 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.quota [None req-02b3ff3d-a872-4015-bc59-04e7961d2bdb tempest-TestMinimumBasicScenario-1447275400 tempest-TestMinimumBasicScenario-1447275400-project] Getting quotas for project 7dd5f20c4a714ee3b555d4aef8b87730. Resources: {'cores', 'ram', 'instances'} {{(pid=120864) _get_quotas /opt/stack/nova/nova/quota.py:393}} Aug 27 02:42:57.460747 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.quota [None req-02b3ff3d-a872-4015-bc59-04e7961d2bdb tempest-TestMinimumBasicScenario-1447275400 tempest-TestMinimumBasicScenario-1447275400-project] Getting quotas for user ba73716667184588b522bc7856404be8 and project 7dd5f20c4a714ee3b555d4aef8b87730. Resources: {'cores', 'ram', 'instances'} {{(pid=120864) _get_quotas /opt/stack/nova/nova/quota.py:383}} Aug 27 02:42:57.472164 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-02b3ff3d-a872-4015-bc59-04e7961d2bdb tempest-TestMinimumBasicScenario-1447275400 tempest-TestMinimumBasicScenario-1447275400-project] Going to run 1 instances... {{(pid=120864) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Aug 27 02:42:57.474706 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-02b3ff3d-a872-4015-bc59-04e7961d2bdb tempest-TestMinimumBasicScenario-1447275400 tempest-TestMinimumBasicScenario-1447275400-project] [instance: 401ec9c3-4ff3-468a-b334-606c558b8d7a] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='8e8edbe7-973c-4c06-8f7c-349dcb775509',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=120864) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Aug 27 02:42:57.524639 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-02b3ff3d-a872-4015-bc59-04e7961d2bdb tempest-TestMinimumBasicScenario-1447275400 tempest-TestMinimumBasicScenario-1447275400-project] 10.222.0.52 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 0.598853 Aug 27 02:42:57.525476 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 608/1209] 10.222.0.52 () {60 vars in 1147 bytes} [Sat Aug 27 02:42:56 2022] POST /compute/v2.1/servers => generated 380 bytes in 601 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Aug 27 02:42:57.536687 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-8547be74-6ea8-4c4b-95e4-a77095979288 tempest-TestMinimumBasicScenario-1447275400 tempest-TestMinimumBasicScenario-1447275400-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:42:57.537326 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-8547be74-6ea8-4c4b-95e4-a77095979288 tempest-TestMinimumBasicScenario-1447275400 tempest-TestMinimumBasicScenario-1447275400-project] [instance: 401ec9c3-4ff3-468a-b334-606c558b8d7a] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:42:57.594468 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-8547be74-6ea8-4c4b-95e4-a77095979288 tempest-TestMinimumBasicScenario-1447275400 tempest-TestMinimumBasicScenario-1447275400-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ba73716667184588b522bc7856404be8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7dd5f20c4a714ee3b555d4aef8b87730', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:42:57.604353 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-8547be74-6ea8-4c4b-95e4-a77095979288 tempest-TestMinimumBasicScenario-1447275400 tempest-TestMinimumBasicScenario-1447275400-project] 10.222.0.52 "GET /compute/v2.1/servers/401ec9c3-4ff3-468a-b334-606c558b8d7a" status: 200 len: 1258 microversion: 2.1 time: 0.071916 Aug 27 02:42:57.605030 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 602/1210] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:42:57 2022] GET /compute/v2.1/servers/401ec9c3-4ff3-468a-b334-606c558b8d7a => generated 1258 bytes in 73 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:42:58.627982 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-fc844098-07a0-41d3-9ca4-e5a2d0f74bb1 tempest-TestMinimumBasicScenario-1447275400 tempest-TestMinimumBasicScenario-1447275400-project] Calling method '>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:42:58.628662 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-fc844098-07a0-41d3-9ca4-e5a2d0f74bb1 tempest-TestMinimumBasicScenario-1447275400 tempest-TestMinimumBasicScenario-1447275400-project] [instance: 401ec9c3-4ff3-468a-b334-606c558b8d7a] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:42:58.638196 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-fc844098-07a0-41d3-9ca4-e5a2d0f74bb1 tempest-TestMinimumBasicScenario-1447275400 tempest-TestMinimumBasicScenario-1447275400-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:42:58.638706 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-fc844098-07a0-41d3-9ca4-e5a2d0f74bb1 tempest-TestMinimumBasicScenario-1447275400 tempest-TestMinimumBasicScenario-1447275400-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:42:58.639178 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-fc844098-07a0-41d3-9ca4-e5a2d0f74bb1 tempest-TestMinimumBasicScenario-1447275400 tempest-TestMinimumBasicScenario-1447275400-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:42:58.639878 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-fc844098-07a0-41d3-9ca4-e5a2d0f74bb1 tempest-TestMinimumBasicScenario-1447275400 tempest-TestMinimumBasicScenario-1447275400-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:42:58.640319 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-fc844098-07a0-41d3-9ca4-e5a2d0f74bb1 tempest-TestMinimumBasicScenario-1447275400 tempest-TestMinimumBasicScenario-1447275400-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:42:58.640737 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-fc844098-07a0-41d3-9ca4-e5a2d0f74bb1 tempest-TestMinimumBasicScenario-1447275400 tempest-TestMinimumBasicScenario-1447275400-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:42:58.705629 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-fc844098-07a0-41d3-9ca4-e5a2d0f74bb1 tempest-TestMinimumBasicScenario-1447275400 tempest-TestMinimumBasicScenario-1447275400-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ba73716667184588b522bc7856404be8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7dd5f20c4a714ee3b555d4aef8b87730', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:42:58.718550 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-fc844098-07a0-41d3-9ca4-e5a2d0f74bb1 tempest-TestMinimumBasicScenario-1447275400 tempest-TestMinimumBasicScenario-1447275400-project] 10.222.0.52 "GET /compute/v2.1/servers/401ec9c3-4ff3-468a-b334-606c558b8d7a" status: 200 len: 1310 microversion: 2.1 time: 0.094440 Aug 27 02:42:58.719469 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 609/1211] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:42:58 2022] GET /compute/v2.1/servers/401ec9c3-4ff3-468a-b334-606c558b8d7a => generated 1310 bytes in 96 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:42:59.748799 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-8c76a4d2-bfcb-441a-81c4-831ad0a85d4f tempest-TestMinimumBasicScenario-1447275400 tempest-TestMinimumBasicScenario-1447275400-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:42:59.750133 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-8c76a4d2-bfcb-441a-81c4-831ad0a85d4f tempest-TestMinimumBasicScenario-1447275400 tempest-TestMinimumBasicScenario-1447275400-project] [instance: 401ec9c3-4ff3-468a-b334-606c558b8d7a] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:42:59.759002 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-8c76a4d2-bfcb-441a-81c4-831ad0a85d4f tempest-TestMinimumBasicScenario-1447275400 tempest-TestMinimumBasicScenario-1447275400-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:42:59.759219 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-8c76a4d2-bfcb-441a-81c4-831ad0a85d4f tempest-TestMinimumBasicScenario-1447275400 tempest-TestMinimumBasicScenario-1447275400-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:42:59.759683 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-8c76a4d2-bfcb-441a-81c4-831ad0a85d4f tempest-TestMinimumBasicScenario-1447275400 tempest-TestMinimumBasicScenario-1447275400-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:42:59.760419 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-8c76a4d2-bfcb-441a-81c4-831ad0a85d4f tempest-TestMinimumBasicScenario-1447275400 tempest-TestMinimumBasicScenario-1447275400-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:42:59.760897 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-8c76a4d2-bfcb-441a-81c4-831ad0a85d4f tempest-TestMinimumBasicScenario-1447275400 tempest-TestMinimumBasicScenario-1447275400-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:42:59.761387 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-8c76a4d2-bfcb-441a-81c4-831ad0a85d4f tempest-TestMinimumBasicScenario-1447275400 tempest-TestMinimumBasicScenario-1447275400-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:42:59.919964 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-8c76a4d2-bfcb-441a-81c4-831ad0a85d4f tempest-TestMinimumBasicScenario-1447275400 tempest-TestMinimumBasicScenario-1447275400-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ba73716667184588b522bc7856404be8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7dd5f20c4a714ee3b555d4aef8b87730', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:42:59.930415 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-8c76a4d2-bfcb-441a-81c4-831ad0a85d4f tempest-TestMinimumBasicScenario-1447275400 tempest-TestMinimumBasicScenario-1447275400-project] 10.222.0.52 "GET /compute/v2.1/servers/401ec9c3-4ff3-468a-b334-606c558b8d7a" status: 200 len: 1358 microversion: 2.1 time: 0.189316 Aug 27 02:42:59.931084 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 603/1212] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:42:59 2022] GET /compute/v2.1/servers/401ec9c3-4ff3-468a-b334-606c558b8d7a => generated 1358 bytes in 191 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:43:00.951885 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-5f0f9f44-45f3-4b68-b955-ea926ddc2c9d tempest-TestMinimumBasicScenario-1447275400 tempest-TestMinimumBasicScenario-1447275400-project] Calling method '>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:43:00.952498 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-5f0f9f44-45f3-4b68-b955-ea926ddc2c9d tempest-TestMinimumBasicScenario-1447275400 tempest-TestMinimumBasicScenario-1447275400-project] [instance: 401ec9c3-4ff3-468a-b334-606c558b8d7a] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:43:00.960028 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-5f0f9f44-45f3-4b68-b955-ea926ddc2c9d tempest-TestMinimumBasicScenario-1447275400 tempest-TestMinimumBasicScenario-1447275400-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:43:00.960514 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-5f0f9f44-45f3-4b68-b955-ea926ddc2c9d tempest-TestMinimumBasicScenario-1447275400 tempest-TestMinimumBasicScenario-1447275400-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:43:00.960934 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-5f0f9f44-45f3-4b68-b955-ea926ddc2c9d tempest-TestMinimumBasicScenario-1447275400 tempest-TestMinimumBasicScenario-1447275400-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:43:00.961665 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-5f0f9f44-45f3-4b68-b955-ea926ddc2c9d tempest-TestMinimumBasicScenario-1447275400 tempest-TestMinimumBasicScenario-1447275400-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:43:00.962371 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-5f0f9f44-45f3-4b68-b955-ea926ddc2c9d tempest-TestMinimumBasicScenario-1447275400 tempest-TestMinimumBasicScenario-1447275400-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:43:00.962991 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-5f0f9f44-45f3-4b68-b955-ea926ddc2c9d tempest-TestMinimumBasicScenario-1447275400 tempest-TestMinimumBasicScenario-1447275400-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:43:01.106609 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-5f0f9f44-45f3-4b68-b955-ea926ddc2c9d tempest-TestMinimumBasicScenario-1447275400 tempest-TestMinimumBasicScenario-1447275400-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ba73716667184588b522bc7856404be8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7dd5f20c4a714ee3b555d4aef8b87730', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:43:01.116161 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-5f0f9f44-45f3-4b68-b955-ea926ddc2c9d tempest-TestMinimumBasicScenario-1447275400 tempest-TestMinimumBasicScenario-1447275400-project] 10.222.0.52 "GET /compute/v2.1/servers/401ec9c3-4ff3-468a-b334-606c558b8d7a" status: 200 len: 1358 microversion: 2.1 time: 0.168044 Aug 27 02:43:01.116757 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 610/1213] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:43:00 2022] GET /compute/v2.1/servers/401ec9c3-4ff3-468a-b334-606c558b8d7a => generated 1358 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:43:02.137603 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-4b558af7-e365-4def-a0a0-63b4f52e021f tempest-TestMinimumBasicScenario-1447275400 tempest-TestMinimumBasicScenario-1447275400-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:43:02.138180 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-4b558af7-e365-4def-a0a0-63b4f52e021f tempest-TestMinimumBasicScenario-1447275400 tempest-TestMinimumBasicScenario-1447275400-project] [instance: 401ec9c3-4ff3-468a-b334-606c558b8d7a] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:43:02.144707 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-4b558af7-e365-4def-a0a0-63b4f52e021f tempest-TestMinimumBasicScenario-1447275400 tempest-TestMinimumBasicScenario-1447275400-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:43:02.145177 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-4b558af7-e365-4def-a0a0-63b4f52e021f tempest-TestMinimumBasicScenario-1447275400 tempest-TestMinimumBasicScenario-1447275400-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:43:02.145617 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-4b558af7-e365-4def-a0a0-63b4f52e021f tempest-TestMinimumBasicScenario-1447275400 tempest-TestMinimumBasicScenario-1447275400-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:43:02.146324 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-4b558af7-e365-4def-a0a0-63b4f52e021f tempest-TestMinimumBasicScenario-1447275400 tempest-TestMinimumBasicScenario-1447275400-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:43:02.146774 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-4b558af7-e365-4def-a0a0-63b4f52e021f tempest-TestMinimumBasicScenario-1447275400 tempest-TestMinimumBasicScenario-1447275400-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:43:02.147196 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-4b558af7-e365-4def-a0a0-63b4f52e021f tempest-TestMinimumBasicScenario-1447275400 tempest-TestMinimumBasicScenario-1447275400-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:43:02.292452 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-4b558af7-e365-4def-a0a0-63b4f52e021f tempest-TestMinimumBasicScenario-1447275400 tempest-TestMinimumBasicScenario-1447275400-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ba73716667184588b522bc7856404be8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7dd5f20c4a714ee3b555d4aef8b87730', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:43:02.300632 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-4b558af7-e365-4def-a0a0-63b4f52e021f tempest-TestMinimumBasicScenario-1447275400 tempest-TestMinimumBasicScenario-1447275400-project] 10.222.0.52 "GET /compute/v2.1/servers/401ec9c3-4ff3-468a-b334-606c558b8d7a" status: 200 len: 1358 microversion: 2.1 time: 0.166997 Aug 27 02:43:02.301735 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 604/1214] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:43:02 2022] GET /compute/v2.1/servers/401ec9c3-4ff3-468a-b334-606c558b8d7a => generated 1358 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:43:03.324398 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-93645e05-1c31-4d31-8f21-86d21371d5dd tempest-TestMinimumBasicScenario-1447275400 tempest-TestMinimumBasicScenario-1447275400-project] Calling method '>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:43:03.324966 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-93645e05-1c31-4d31-8f21-86d21371d5dd tempest-TestMinimumBasicScenario-1447275400 tempest-TestMinimumBasicScenario-1447275400-project] [instance: 401ec9c3-4ff3-468a-b334-606c558b8d7a] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:43:03.332003 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-93645e05-1c31-4d31-8f21-86d21371d5dd tempest-TestMinimumBasicScenario-1447275400 tempest-TestMinimumBasicScenario-1447275400-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:43:03.332479 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-93645e05-1c31-4d31-8f21-86d21371d5dd tempest-TestMinimumBasicScenario-1447275400 tempest-TestMinimumBasicScenario-1447275400-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:43:03.332911 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-93645e05-1c31-4d31-8f21-86d21371d5dd tempest-TestMinimumBasicScenario-1447275400 tempest-TestMinimumBasicScenario-1447275400-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:43:03.333646 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-93645e05-1c31-4d31-8f21-86d21371d5dd tempest-TestMinimumBasicScenario-1447275400 tempest-TestMinimumBasicScenario-1447275400-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:43:03.334098 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-93645e05-1c31-4d31-8f21-86d21371d5dd tempest-TestMinimumBasicScenario-1447275400 tempest-TestMinimumBasicScenario-1447275400-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:43:03.334521 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-93645e05-1c31-4d31-8f21-86d21371d5dd tempest-TestMinimumBasicScenario-1447275400 tempest-TestMinimumBasicScenario-1447275400-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:43:03.483946 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-93645e05-1c31-4d31-8f21-86d21371d5dd tempest-TestMinimumBasicScenario-1447275400 tempest-TestMinimumBasicScenario-1447275400-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ba73716667184588b522bc7856404be8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7dd5f20c4a714ee3b555d4aef8b87730', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:43:03.491783 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-93645e05-1c31-4d31-8f21-86d21371d5dd tempest-TestMinimumBasicScenario-1447275400 tempest-TestMinimumBasicScenario-1447275400-project] 10.222.0.52 "GET /compute/v2.1/servers/401ec9c3-4ff3-468a-b334-606c558b8d7a" status: 200 len: 1358 microversion: 2.1 time: 0.171467 Aug 27 02:43:03.492346 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 611/1215] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:43:03 2022] GET /compute/v2.1/servers/401ec9c3-4ff3-468a-b334-606c558b8d7a => generated 1358 bytes in 173 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:43:04.513356 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-784b006f-3fad-4e51-ae4e-3a51dba6400c tempest-TestMinimumBasicScenario-1447275400 tempest-TestMinimumBasicScenario-1447275400-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:43:04.514027 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-784b006f-3fad-4e51-ae4e-3a51dba6400c tempest-TestMinimumBasicScenario-1447275400 tempest-TestMinimumBasicScenario-1447275400-project] [instance: 401ec9c3-4ff3-468a-b334-606c558b8d7a] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:43:04.521353 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-784b006f-3fad-4e51-ae4e-3a51dba6400c tempest-TestMinimumBasicScenario-1447275400 tempest-TestMinimumBasicScenario-1447275400-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:43:04.521837 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-784b006f-3fad-4e51-ae4e-3a51dba6400c tempest-TestMinimumBasicScenario-1447275400 tempest-TestMinimumBasicScenario-1447275400-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:43:04.522267 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-784b006f-3fad-4e51-ae4e-3a51dba6400c tempest-TestMinimumBasicScenario-1447275400 tempest-TestMinimumBasicScenario-1447275400-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:43:04.522969 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-784b006f-3fad-4e51-ae4e-3a51dba6400c tempest-TestMinimumBasicScenario-1447275400 tempest-TestMinimumBasicScenario-1447275400-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:43:04.523419 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-784b006f-3fad-4e51-ae4e-3a51dba6400c tempest-TestMinimumBasicScenario-1447275400 tempest-TestMinimumBasicScenario-1447275400-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:43:04.523837 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-784b006f-3fad-4e51-ae4e-3a51dba6400c tempest-TestMinimumBasicScenario-1447275400 tempest-TestMinimumBasicScenario-1447275400-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:43:04.677155 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-784b006f-3fad-4e51-ae4e-3a51dba6400c tempest-TestMinimumBasicScenario-1447275400 tempest-TestMinimumBasicScenario-1447275400-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ba73716667184588b522bc7856404be8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7dd5f20c4a714ee3b555d4aef8b87730', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:43:04.685546 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-784b006f-3fad-4e51-ae4e-3a51dba6400c tempest-TestMinimumBasicScenario-1447275400 tempest-TestMinimumBasicScenario-1447275400-project] 10.222.0.52 "GET /compute/v2.1/servers/401ec9c3-4ff3-468a-b334-606c558b8d7a" status: 200 len: 1358 microversion: 2.1 time: 0.176099 Aug 27 02:43:04.693793 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 605/1216] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:43:04 2022] GET /compute/v2.1/servers/401ec9c3-4ff3-468a-b334-606c558b8d7a => generated 1358 bytes in 185 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:43:05.707482 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-4a61961d-00e3-4411-aec2-f88a3b713e28 tempest-TestMinimumBasicScenario-1447275400 tempest-TestMinimumBasicScenario-1447275400-project] Calling method '>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:43:05.708201 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-4a61961d-00e3-4411-aec2-f88a3b713e28 tempest-TestMinimumBasicScenario-1447275400 tempest-TestMinimumBasicScenario-1447275400-project] [instance: 401ec9c3-4ff3-468a-b334-606c558b8d7a] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:43:05.714810 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-4a61961d-00e3-4411-aec2-f88a3b713e28 tempest-TestMinimumBasicScenario-1447275400 tempest-TestMinimumBasicScenario-1447275400-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:43:05.715328 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-4a61961d-00e3-4411-aec2-f88a3b713e28 tempest-TestMinimumBasicScenario-1447275400 tempest-TestMinimumBasicScenario-1447275400-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:43:05.715716 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-4a61961d-00e3-4411-aec2-f88a3b713e28 tempest-TestMinimumBasicScenario-1447275400 tempest-TestMinimumBasicScenario-1447275400-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:43:05.717368 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-4a61961d-00e3-4411-aec2-f88a3b713e28 tempest-TestMinimumBasicScenario-1447275400 tempest-TestMinimumBasicScenario-1447275400-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:43:05.717368 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-4a61961d-00e3-4411-aec2-f88a3b713e28 tempest-TestMinimumBasicScenario-1447275400 tempest-TestMinimumBasicScenario-1447275400-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:43:05.717635 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-4a61961d-00e3-4411-aec2-f88a3b713e28 tempest-TestMinimumBasicScenario-1447275400 tempest-TestMinimumBasicScenario-1447275400-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:43:05.861026 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-4a61961d-00e3-4411-aec2-f88a3b713e28 tempest-TestMinimumBasicScenario-1447275400 tempest-TestMinimumBasicScenario-1447275400-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ba73716667184588b522bc7856404be8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7dd5f20c4a714ee3b555d4aef8b87730', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:43:05.870189 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-4a61961d-00e3-4411-aec2-f88a3b713e28 tempest-TestMinimumBasicScenario-1447275400 tempest-TestMinimumBasicScenario-1447275400-project] 10.222.0.52 "GET /compute/v2.1/servers/401ec9c3-4ff3-468a-b334-606c558b8d7a" status: 200 len: 1358 microversion: 2.1 time: 0.167415 Aug 27 02:43:05.870766 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 612/1217] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:43:05 2022] GET /compute/v2.1/servers/401ec9c3-4ff3-468a-b334-606c558b8d7a => generated 1358 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:43:06.892608 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-3d9c29c5-0c15-4261-8f5e-38159a8464e6 tempest-TestMinimumBasicScenario-1447275400 tempest-TestMinimumBasicScenario-1447275400-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:43:06.893528 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-3d9c29c5-0c15-4261-8f5e-38159a8464e6 tempest-TestMinimumBasicScenario-1447275400 tempest-TestMinimumBasicScenario-1447275400-project] [instance: 401ec9c3-4ff3-468a-b334-606c558b8d7a] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:43:06.911308 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-3d9c29c5-0c15-4261-8f5e-38159a8464e6 tempest-TestMinimumBasicScenario-1447275400 tempest-TestMinimumBasicScenario-1447275400-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:43:06.911821 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-3d9c29c5-0c15-4261-8f5e-38159a8464e6 tempest-TestMinimumBasicScenario-1447275400 tempest-TestMinimumBasicScenario-1447275400-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:43:06.912228 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-3d9c29c5-0c15-4261-8f5e-38159a8464e6 tempest-TestMinimumBasicScenario-1447275400 tempest-TestMinimumBasicScenario-1447275400-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:43:06.912955 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-3d9c29c5-0c15-4261-8f5e-38159a8464e6 tempest-TestMinimumBasicScenario-1447275400 tempest-TestMinimumBasicScenario-1447275400-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:43:06.913476 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-3d9c29c5-0c15-4261-8f5e-38159a8464e6 tempest-TestMinimumBasicScenario-1447275400 tempest-TestMinimumBasicScenario-1447275400-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:43:06.913856 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-3d9c29c5-0c15-4261-8f5e-38159a8464e6 tempest-TestMinimumBasicScenario-1447275400 tempest-TestMinimumBasicScenario-1447275400-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:43:07.055457 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-3d9c29c5-0c15-4261-8f5e-38159a8464e6 tempest-TestMinimumBasicScenario-1447275400 tempest-TestMinimumBasicScenario-1447275400-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ba73716667184588b522bc7856404be8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7dd5f20c4a714ee3b555d4aef8b87730', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:43:07.063934 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-3d9c29c5-0c15-4261-8f5e-38159a8464e6 tempest-TestMinimumBasicScenario-1447275400 tempest-TestMinimumBasicScenario-1447275400-project] 10.222.0.52 "GET /compute/v2.1/servers/401ec9c3-4ff3-468a-b334-606c558b8d7a" status: 200 len: 1358 microversion: 2.1 time: 0.176031 Aug 27 02:43:07.064632 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 606/1218] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:43:06 2022] GET /compute/v2.1/servers/401ec9c3-4ff3-468a-b334-606c558b8d7a => generated 1358 bytes in 177 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:43:08.085378 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-eaa95b8d-c736-406d-bb05-a1035db7ca02 tempest-TestMinimumBasicScenario-1447275400 tempest-TestMinimumBasicScenario-1447275400-project] Calling method '>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:43:08.086072 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-eaa95b8d-c736-406d-bb05-a1035db7ca02 tempest-TestMinimumBasicScenario-1447275400 tempest-TestMinimumBasicScenario-1447275400-project] [instance: 401ec9c3-4ff3-468a-b334-606c558b8d7a] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:43:08.088601 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG dbcounter [-] [120864] Writing DB stats nova_cell0:SELECT=2 {{(pid=120864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 27 02:43:08.089273 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG dbcounter [-] [120864] Writing DB stats nova_cell0:SELECT=6 {{(pid=120864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 27 02:43:08.095040 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-eaa95b8d-c736-406d-bb05-a1035db7ca02 tempest-TestMinimumBasicScenario-1447275400 tempest-TestMinimumBasicScenario-1447275400-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:43:08.095518 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-eaa95b8d-c736-406d-bb05-a1035db7ca02 tempest-TestMinimumBasicScenario-1447275400 tempest-TestMinimumBasicScenario-1447275400-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:43:08.095946 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-eaa95b8d-c736-406d-bb05-a1035db7ca02 tempest-TestMinimumBasicScenario-1447275400 tempest-TestMinimumBasicScenario-1447275400-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:43:08.096938 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-eaa95b8d-c736-406d-bb05-a1035db7ca02 tempest-TestMinimumBasicScenario-1447275400 tempest-TestMinimumBasicScenario-1447275400-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:43:08.097560 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-eaa95b8d-c736-406d-bb05-a1035db7ca02 tempest-TestMinimumBasicScenario-1447275400 tempest-TestMinimumBasicScenario-1447275400-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:43:08.097982 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-eaa95b8d-c736-406d-bb05-a1035db7ca02 tempest-TestMinimumBasicScenario-1447275400 tempest-TestMinimumBasicScenario-1447275400-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:43:08.236183 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-eaa95b8d-c736-406d-bb05-a1035db7ca02 tempest-TestMinimumBasicScenario-1447275400 tempest-TestMinimumBasicScenario-1447275400-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ba73716667184588b522bc7856404be8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7dd5f20c4a714ee3b555d4aef8b87730', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:43:08.244487 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-eaa95b8d-c736-406d-bb05-a1035db7ca02 tempest-TestMinimumBasicScenario-1447275400 tempest-TestMinimumBasicScenario-1447275400-project] 10.222.0.52 "GET /compute/v2.1/servers/401ec9c3-4ff3-468a-b334-606c558b8d7a" status: 200 len: 1358 microversion: 2.1 time: 0.163370 Aug 27 02:43:08.245055 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 613/1219] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:43:08 2022] GET /compute/v2.1/servers/401ec9c3-4ff3-468a-b334-606c558b8d7a => generated 1358 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:43:09.275518 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-b8841d1a-06b5-4cfd-abc3-080cb8250b26 tempest-TestMinimumBasicScenario-1447275400 tempest-TestMinimumBasicScenario-1447275400-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:43:09.276208 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-b8841d1a-06b5-4cfd-abc3-080cb8250b26 tempest-TestMinimumBasicScenario-1447275400 tempest-TestMinimumBasicScenario-1447275400-project] [instance: 401ec9c3-4ff3-468a-b334-606c558b8d7a] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:43:09.278918 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG dbcounter [-] [120863] Writing DB stats nova_cell0:SELECT=6 {{(pid=120863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 27 02:43:09.286180 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-b8841d1a-06b5-4cfd-abc3-080cb8250b26 tempest-TestMinimumBasicScenario-1447275400 tempest-TestMinimumBasicScenario-1447275400-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:43:09.286657 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-b8841d1a-06b5-4cfd-abc3-080cb8250b26 tempest-TestMinimumBasicScenario-1447275400 tempest-TestMinimumBasicScenario-1447275400-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:43:09.287087 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-b8841d1a-06b5-4cfd-abc3-080cb8250b26 tempest-TestMinimumBasicScenario-1447275400 tempest-TestMinimumBasicScenario-1447275400-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:43:09.287769 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-b8841d1a-06b5-4cfd-abc3-080cb8250b26 tempest-TestMinimumBasicScenario-1447275400 tempest-TestMinimumBasicScenario-1447275400-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:43:09.288212 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-b8841d1a-06b5-4cfd-abc3-080cb8250b26 tempest-TestMinimumBasicScenario-1447275400 tempest-TestMinimumBasicScenario-1447275400-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:43:09.288628 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-b8841d1a-06b5-4cfd-abc3-080cb8250b26 tempest-TestMinimumBasicScenario-1447275400 tempest-TestMinimumBasicScenario-1447275400-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:43:09.427192 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-b8841d1a-06b5-4cfd-abc3-080cb8250b26 tempest-TestMinimumBasicScenario-1447275400 tempest-TestMinimumBasicScenario-1447275400-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ba73716667184588b522bc7856404be8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7dd5f20c4a714ee3b555d4aef8b87730', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:43:09.439616 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-b8841d1a-06b5-4cfd-abc3-080cb8250b26 tempest-TestMinimumBasicScenario-1447275400 tempest-TestMinimumBasicScenario-1447275400-project] 10.222.0.52 "GET /compute/v2.1/servers/401ec9c3-4ff3-468a-b334-606c558b8d7a" status: 200 len: 1358 microversion: 2.1 time: 0.169219 Aug 27 02:43:09.440560 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 607/1220] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:43:09 2022] GET /compute/v2.1/servers/401ec9c3-4ff3-468a-b334-606c558b8d7a => generated 1358 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:43:10.462930 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-873a81c3-1f9b-42b2-9361-a3faf1dce13a tempest-TestMinimumBasicScenario-1447275400 tempest-TestMinimumBasicScenario-1447275400-project] Calling method '>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:43:10.463721 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-873a81c3-1f9b-42b2-9361-a3faf1dce13a tempest-TestMinimumBasicScenario-1447275400 tempest-TestMinimumBasicScenario-1447275400-project] [instance: 401ec9c3-4ff3-468a-b334-606c558b8d7a] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:43:10.471926 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-873a81c3-1f9b-42b2-9361-a3faf1dce13a tempest-TestMinimumBasicScenario-1447275400 tempest-TestMinimumBasicScenario-1447275400-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:43:10.472712 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-873a81c3-1f9b-42b2-9361-a3faf1dce13a tempest-TestMinimumBasicScenario-1447275400 tempest-TestMinimumBasicScenario-1447275400-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:43:10.473293 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-873a81c3-1f9b-42b2-9361-a3faf1dce13a tempest-TestMinimumBasicScenario-1447275400 tempest-TestMinimumBasicScenario-1447275400-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:43:10.474144 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-873a81c3-1f9b-42b2-9361-a3faf1dce13a tempest-TestMinimumBasicScenario-1447275400 tempest-TestMinimumBasicScenario-1447275400-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:43:10.474693 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-873a81c3-1f9b-42b2-9361-a3faf1dce13a tempest-TestMinimumBasicScenario-1447275400 tempest-TestMinimumBasicScenario-1447275400-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:43:10.475188 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-873a81c3-1f9b-42b2-9361-a3faf1dce13a tempest-TestMinimumBasicScenario-1447275400 tempest-TestMinimumBasicScenario-1447275400-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:43:10.621151 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-873a81c3-1f9b-42b2-9361-a3faf1dce13a tempest-TestMinimumBasicScenario-1447275400 tempest-TestMinimumBasicScenario-1447275400-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ba73716667184588b522bc7856404be8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7dd5f20c4a714ee3b555d4aef8b87730', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:43:10.628940 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-873a81c3-1f9b-42b2-9361-a3faf1dce13a tempest-TestMinimumBasicScenario-1447275400 tempest-TestMinimumBasicScenario-1447275400-project] 10.222.0.52 "GET /compute/v2.1/servers/401ec9c3-4ff3-468a-b334-606c558b8d7a" status: 200 len: 1358 microversion: 2.1 time: 0.170568 Aug 27 02:43:10.629513 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 614/1221] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:43:10 2022] GET /compute/v2.1/servers/401ec9c3-4ff3-468a-b334-606c558b8d7a => generated 1358 bytes in 172 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:43:11.334037 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [req-1c490043-b99a-40f7-8278-7a5134b526ac req-2f37e632-a7b1-4e2d-a6a1-81078dae1909 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "401ec9c3-4ff3-468a-b334-606c558b8d7a", "tag": "974dc3f6-d8bf-4ab5-a284-a4ecb61b8866"}]} {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 27 02:43:11.344311 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [req-1c490043-b99a-40f7-8278-7a5134b526ac req-2f37e632-a7b1-4e2d-a6a1-81078dae1909 service nova] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:43:11.344788 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [req-1c490043-b99a-40f7-8278-7a5134b526ac req-2f37e632-a7b1-4e2d-a6a1-81078dae1909 service nova] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:43:11.345266 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [req-1c490043-b99a-40f7-8278-7a5134b526ac req-2f37e632-a7b1-4e2d-a6a1-81078dae1909 service nova] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:43:11.357411 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.compute.server_external_events [req-1c490043-b99a-40f7-8278-7a5134b526ac req-2f37e632-a7b1-4e2d-a6a1-81078dae1909 service nova] Creating event network-changed:974dc3f6-d8bf-4ab5-a284-a4ecb61b8866 for instance 401ec9c3-4ff3-468a-b334-606c558b8d7a on hv-cinder-84630 Aug 27 02:43:11.362752 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [req-1c490043-b99a-40f7-8278-7a5134b526ac req-2f37e632-a7b1-4e2d-a6a1-81078dae1909 service nova] 10.222.0.52 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.032839 Aug 27 02:43:11.363395 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 608/1222] 10.222.0.52 () {64 vars in 1317 bytes} [Sat Aug 27 02:43:11 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 35 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 27 02:43:11.650109 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-a32ec25b-5fdf-4c36-b247-754a0247483d tempest-TestMinimumBasicScenario-1447275400 tempest-TestMinimumBasicScenario-1447275400-project] Calling method '>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:43:11.650722 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-a32ec25b-5fdf-4c36-b247-754a0247483d tempest-TestMinimumBasicScenario-1447275400 tempest-TestMinimumBasicScenario-1447275400-project] [instance: 401ec9c3-4ff3-468a-b334-606c558b8d7a] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:43:11.657636 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-a32ec25b-5fdf-4c36-b247-754a0247483d tempest-TestMinimumBasicScenario-1447275400 tempest-TestMinimumBasicScenario-1447275400-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:43:11.658130 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-a32ec25b-5fdf-4c36-b247-754a0247483d tempest-TestMinimumBasicScenario-1447275400 tempest-TestMinimumBasicScenario-1447275400-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:43:11.658567 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-a32ec25b-5fdf-4c36-b247-754a0247483d tempest-TestMinimumBasicScenario-1447275400 tempest-TestMinimumBasicScenario-1447275400-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:43:11.659796 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-a32ec25b-5fdf-4c36-b247-754a0247483d tempest-TestMinimumBasicScenario-1447275400 tempest-TestMinimumBasicScenario-1447275400-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:43:11.660378 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-a32ec25b-5fdf-4c36-b247-754a0247483d tempest-TestMinimumBasicScenario-1447275400 tempest-TestMinimumBasicScenario-1447275400-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:43:11.660808 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-a32ec25b-5fdf-4c36-b247-754a0247483d tempest-TestMinimumBasicScenario-1447275400 tempest-TestMinimumBasicScenario-1447275400-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:43:11.839667 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-a32ec25b-5fdf-4c36-b247-754a0247483d tempest-TestMinimumBasicScenario-1447275400 tempest-TestMinimumBasicScenario-1447275400-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ba73716667184588b522bc7856404be8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7dd5f20c4a714ee3b555d4aef8b87730', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:43:11.848066 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-a32ec25b-5fdf-4c36-b247-754a0247483d tempest-TestMinimumBasicScenario-1447275400 tempest-TestMinimumBasicScenario-1447275400-project] 10.222.0.52 "GET /compute/v2.1/servers/401ec9c3-4ff3-468a-b334-606c558b8d7a" status: 200 len: 1358 microversion: 2.1 time: 0.201743 Aug 27 02:43:11.848590 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 615/1223] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:43:11 2022] GET /compute/v2.1/servers/401ec9c3-4ff3-468a-b334-606c558b8d7a => generated 1358 bytes in 203 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:43:12.878387 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-66d2bebb-d249-456d-9fc1-e36d54b9020c tempest-TestMinimumBasicScenario-1447275400 tempest-TestMinimumBasicScenario-1447275400-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:43:12.879341 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-66d2bebb-d249-456d-9fc1-e36d54b9020c tempest-TestMinimumBasicScenario-1447275400 tempest-TestMinimumBasicScenario-1447275400-project] [instance: 401ec9c3-4ff3-468a-b334-606c558b8d7a] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:43:12.888042 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-66d2bebb-d249-456d-9fc1-e36d54b9020c tempest-TestMinimumBasicScenario-1447275400 tempest-TestMinimumBasicScenario-1447275400-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:43:12.888708 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-66d2bebb-d249-456d-9fc1-e36d54b9020c tempest-TestMinimumBasicScenario-1447275400 tempest-TestMinimumBasicScenario-1447275400-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:43:12.889384 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-66d2bebb-d249-456d-9fc1-e36d54b9020c tempest-TestMinimumBasicScenario-1447275400 tempest-TestMinimumBasicScenario-1447275400-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:43:12.890320 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-66d2bebb-d249-456d-9fc1-e36d54b9020c tempest-TestMinimumBasicScenario-1447275400 tempest-TestMinimumBasicScenario-1447275400-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:43:12.890992 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-66d2bebb-d249-456d-9fc1-e36d54b9020c tempest-TestMinimumBasicScenario-1447275400 tempest-TestMinimumBasicScenario-1447275400-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:43:12.891583 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-66d2bebb-d249-456d-9fc1-e36d54b9020c tempest-TestMinimumBasicScenario-1447275400 tempest-TestMinimumBasicScenario-1447275400-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:43:13.041340 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-66d2bebb-d249-456d-9fc1-e36d54b9020c tempest-TestMinimumBasicScenario-1447275400 tempest-TestMinimumBasicScenario-1447275400-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ba73716667184588b522bc7856404be8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7dd5f20c4a714ee3b555d4aef8b87730', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:43:13.049164 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-66d2bebb-d249-456d-9fc1-e36d54b9020c tempest-TestMinimumBasicScenario-1447275400 tempest-TestMinimumBasicScenario-1447275400-project] 10.222.0.52 "GET /compute/v2.1/servers/401ec9c3-4ff3-468a-b334-606c558b8d7a" status: 200 len: 1358 microversion: 2.1 time: 0.177380 Aug 27 02:43:13.049757 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 609/1224] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:43:12 2022] GET /compute/v2.1/servers/401ec9c3-4ff3-468a-b334-606c558b8d7a => generated 1358 bytes in 179 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:43:14.070352 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-9a96b74f-aeb6-4971-84ec-0b728836ccc5 tempest-TestMinimumBasicScenario-1447275400 tempest-TestMinimumBasicScenario-1447275400-project] Calling method '>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:43:14.070972 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-9a96b74f-aeb6-4971-84ec-0b728836ccc5 tempest-TestMinimumBasicScenario-1447275400 tempest-TestMinimumBasicScenario-1447275400-project] [instance: 401ec9c3-4ff3-468a-b334-606c558b8d7a] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:43:14.079687 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-9a96b74f-aeb6-4971-84ec-0b728836ccc5 tempest-TestMinimumBasicScenario-1447275400 tempest-TestMinimumBasicScenario-1447275400-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:43:14.080254 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-9a96b74f-aeb6-4971-84ec-0b728836ccc5 tempest-TestMinimumBasicScenario-1447275400 tempest-TestMinimumBasicScenario-1447275400-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:43:14.080748 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-9a96b74f-aeb6-4971-84ec-0b728836ccc5 tempest-TestMinimumBasicScenario-1447275400 tempest-TestMinimumBasicScenario-1447275400-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:43:14.081579 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-9a96b74f-aeb6-4971-84ec-0b728836ccc5 tempest-TestMinimumBasicScenario-1447275400 tempest-TestMinimumBasicScenario-1447275400-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:43:14.082028 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-9a96b74f-aeb6-4971-84ec-0b728836ccc5 tempest-TestMinimumBasicScenario-1447275400 tempest-TestMinimumBasicScenario-1447275400-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:43:14.082440 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-9a96b74f-aeb6-4971-84ec-0b728836ccc5 tempest-TestMinimumBasicScenario-1447275400 tempest-TestMinimumBasicScenario-1447275400-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:43:14.234806 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-9a96b74f-aeb6-4971-84ec-0b728836ccc5 tempest-TestMinimumBasicScenario-1447275400 tempest-TestMinimumBasicScenario-1447275400-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ba73716667184588b522bc7856404be8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7dd5f20c4a714ee3b555d4aef8b87730', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:43:14.243189 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-9a96b74f-aeb6-4971-84ec-0b728836ccc5 tempest-TestMinimumBasicScenario-1447275400 tempest-TestMinimumBasicScenario-1447275400-project] 10.222.0.52 "GET /compute/v2.1/servers/401ec9c3-4ff3-468a-b334-606c558b8d7a" status: 200 len: 1358 microversion: 2.1 time: 0.176752 Aug 27 02:43:14.243810 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 616/1225] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:43:14 2022] GET /compute/v2.1/servers/401ec9c3-4ff3-468a-b334-606c558b8d7a => generated 1358 bytes in 178 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:43:15.268704 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-a0d9d213-ff25-46d0-9c42-a8f1cc31a9e0 tempest-TestMinimumBasicScenario-1447275400 tempest-TestMinimumBasicScenario-1447275400-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:43:15.269584 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-a0d9d213-ff25-46d0-9c42-a8f1cc31a9e0 tempest-TestMinimumBasicScenario-1447275400 tempest-TestMinimumBasicScenario-1447275400-project] [instance: 401ec9c3-4ff3-468a-b334-606c558b8d7a] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:43:15.280022 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-a0d9d213-ff25-46d0-9c42-a8f1cc31a9e0 tempest-TestMinimumBasicScenario-1447275400 tempest-TestMinimumBasicScenario-1447275400-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:43:15.280836 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-a0d9d213-ff25-46d0-9c42-a8f1cc31a9e0 tempest-TestMinimumBasicScenario-1447275400 tempest-TestMinimumBasicScenario-1447275400-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:43:15.281619 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-a0d9d213-ff25-46d0-9c42-a8f1cc31a9e0 tempest-TestMinimumBasicScenario-1447275400 tempest-TestMinimumBasicScenario-1447275400-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:43:15.282780 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-a0d9d213-ff25-46d0-9c42-a8f1cc31a9e0 tempest-TestMinimumBasicScenario-1447275400 tempest-TestMinimumBasicScenario-1447275400-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:43:15.283538 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-a0d9d213-ff25-46d0-9c42-a8f1cc31a9e0 tempest-TestMinimumBasicScenario-1447275400 tempest-TestMinimumBasicScenario-1447275400-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:43:15.284230 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-a0d9d213-ff25-46d0-9c42-a8f1cc31a9e0 tempest-TestMinimumBasicScenario-1447275400 tempest-TestMinimumBasicScenario-1447275400-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:43:15.438972 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-a0d9d213-ff25-46d0-9c42-a8f1cc31a9e0 tempest-TestMinimumBasicScenario-1447275400 tempest-TestMinimumBasicScenario-1447275400-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ba73716667184588b522bc7856404be8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7dd5f20c4a714ee3b555d4aef8b87730', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:43:15.447415 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-a0d9d213-ff25-46d0-9c42-a8f1cc31a9e0 tempest-TestMinimumBasicScenario-1447275400 tempest-TestMinimumBasicScenario-1447275400-project] 10.222.0.52 "GET /compute/v2.1/servers/401ec9c3-4ff3-468a-b334-606c558b8d7a" status: 200 len: 1358 microversion: 2.1 time: 0.185573 Aug 27 02:43:15.448002 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 610/1226] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:43:15 2022] GET /compute/v2.1/servers/401ec9c3-4ff3-468a-b334-606c558b8d7a => generated 1358 bytes in 187 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:43:16.470201 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-60e715a1-3a32-41e2-ba0b-970b3fb00a73 tempest-TestMinimumBasicScenario-1447275400 tempest-TestMinimumBasicScenario-1447275400-project] Calling method '>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:43:16.470963 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-60e715a1-3a32-41e2-ba0b-970b3fb00a73 tempest-TestMinimumBasicScenario-1447275400 tempest-TestMinimumBasicScenario-1447275400-project] [instance: 401ec9c3-4ff3-468a-b334-606c558b8d7a] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:43:16.481602 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-60e715a1-3a32-41e2-ba0b-970b3fb00a73 tempest-TestMinimumBasicScenario-1447275400 tempest-TestMinimumBasicScenario-1447275400-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:43:16.482393 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-60e715a1-3a32-41e2-ba0b-970b3fb00a73 tempest-TestMinimumBasicScenario-1447275400 tempest-TestMinimumBasicScenario-1447275400-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:43:16.483090 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-60e715a1-3a32-41e2-ba0b-970b3fb00a73 tempest-TestMinimumBasicScenario-1447275400 tempest-TestMinimumBasicScenario-1447275400-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:43:16.484237 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-60e715a1-3a32-41e2-ba0b-970b3fb00a73 tempest-TestMinimumBasicScenario-1447275400 tempest-TestMinimumBasicScenario-1447275400-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:43:16.484977 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-60e715a1-3a32-41e2-ba0b-970b3fb00a73 tempest-TestMinimumBasicScenario-1447275400 tempest-TestMinimumBasicScenario-1447275400-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:43:16.485721 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-60e715a1-3a32-41e2-ba0b-970b3fb00a73 tempest-TestMinimumBasicScenario-1447275400 tempest-TestMinimumBasicScenario-1447275400-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:43:16.625580 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-60e715a1-3a32-41e2-ba0b-970b3fb00a73 tempest-TestMinimumBasicScenario-1447275400 tempest-TestMinimumBasicScenario-1447275400-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ba73716667184588b522bc7856404be8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7dd5f20c4a714ee3b555d4aef8b87730', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:43:16.633373 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-60e715a1-3a32-41e2-ba0b-970b3fb00a73 tempest-TestMinimumBasicScenario-1447275400 tempest-TestMinimumBasicScenario-1447275400-project] 10.222.0.52 "GET /compute/v2.1/servers/401ec9c3-4ff3-468a-b334-606c558b8d7a" status: 200 len: 1358 microversion: 2.1 time: 0.168940 Aug 27 02:43:16.633934 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 617/1227] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:43:16 2022] GET /compute/v2.1/servers/401ec9c3-4ff3-468a-b334-606c558b8d7a => generated 1358 bytes in 170 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:43:17.015742 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [req-116aa303-f6bf-4189-a01c-ee663f081ac4 req-c40a8f38-2d10-492d-9a27-fada978028b9 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "401ec9c3-4ff3-468a-b334-606c558b8d7a", "name": "network-vif-plugged", "status": "completed", "tag": "974dc3f6-d8bf-4ab5-a284-a4ecb61b8866"}]} {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 27 02:43:17.026473 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [req-116aa303-f6bf-4189-a01c-ee663f081ac4 req-c40a8f38-2d10-492d-9a27-fada978028b9 service nova] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:43:17.026914 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [req-116aa303-f6bf-4189-a01c-ee663f081ac4 req-c40a8f38-2d10-492d-9a27-fada978028b9 service nova] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:43:17.027570 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [req-116aa303-f6bf-4189-a01c-ee663f081ac4 req-c40a8f38-2d10-492d-9a27-fada978028b9 service nova] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:43:17.039967 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.compute.server_external_events [req-116aa303-f6bf-4189-a01c-ee663f081ac4 req-c40a8f38-2d10-492d-9a27-fada978028b9 service nova] Creating event network-vif-plugged:974dc3f6-d8bf-4ab5-a284-a4ecb61b8866 for instance 401ec9c3-4ff3-468a-b334-606c558b8d7a on hv-cinder-84630 Aug 27 02:43:17.045027 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [req-116aa303-f6bf-4189-a01c-ee663f081ac4 req-c40a8f38-2d10-492d-9a27-fada978028b9 service nova] 10.222.0.52 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.033024 Aug 27 02:43:17.045686 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 611/1228] 10.222.0.52 () {64 vars in 1317 bytes} [Sat Aug 27 02:43:17 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 34 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 27 02:43:17.656986 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-637b544d-d2f0-4eec-964f-0c32aa6a2bff tempest-TestMinimumBasicScenario-1447275400 tempest-TestMinimumBasicScenario-1447275400-project] Calling method '>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:43:17.657655 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-637b544d-d2f0-4eec-964f-0c32aa6a2bff tempest-TestMinimumBasicScenario-1447275400 tempest-TestMinimumBasicScenario-1447275400-project] [instance: 401ec9c3-4ff3-468a-b334-606c558b8d7a] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:43:17.665375 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-637b544d-d2f0-4eec-964f-0c32aa6a2bff tempest-TestMinimumBasicScenario-1447275400 tempest-TestMinimumBasicScenario-1447275400-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:43:17.665861 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-637b544d-d2f0-4eec-964f-0c32aa6a2bff tempest-TestMinimumBasicScenario-1447275400 tempest-TestMinimumBasicScenario-1447275400-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:43:17.666294 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-637b544d-d2f0-4eec-964f-0c32aa6a2bff tempest-TestMinimumBasicScenario-1447275400 tempest-TestMinimumBasicScenario-1447275400-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:43:17.666990 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-637b544d-d2f0-4eec-964f-0c32aa6a2bff tempest-TestMinimumBasicScenario-1447275400 tempest-TestMinimumBasicScenario-1447275400-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:43:17.667435 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-637b544d-d2f0-4eec-964f-0c32aa6a2bff tempest-TestMinimumBasicScenario-1447275400 tempest-TestMinimumBasicScenario-1447275400-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:43:17.667845 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-637b544d-d2f0-4eec-964f-0c32aa6a2bff tempest-TestMinimumBasicScenario-1447275400 tempest-TestMinimumBasicScenario-1447275400-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:43:17.812486 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-637b544d-d2f0-4eec-964f-0c32aa6a2bff tempest-TestMinimumBasicScenario-1447275400 tempest-TestMinimumBasicScenario-1447275400-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ba73716667184588b522bc7856404be8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7dd5f20c4a714ee3b555d4aef8b87730', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:43:17.823710 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-637b544d-d2f0-4eec-964f-0c32aa6a2bff tempest-TestMinimumBasicScenario-1447275400 tempest-TestMinimumBasicScenario-1447275400-project] 10.222.0.52 "GET /compute/v2.1/servers/401ec9c3-4ff3-468a-b334-606c558b8d7a" status: 200 len: 1358 microversion: 2.1 time: 0.172143 Aug 27 02:43:17.824614 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 618/1229] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:43:17 2022] GET /compute/v2.1/servers/401ec9c3-4ff3-468a-b334-606c558b8d7a => generated 1358 bytes in 174 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:43:18.847337 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-620c7f9e-cdf2-4317-b586-0fb347f9104f tempest-TestMinimumBasicScenario-1447275400 tempest-TestMinimumBasicScenario-1447275400-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:43:18.848734 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-620c7f9e-cdf2-4317-b586-0fb347f9104f tempest-TestMinimumBasicScenario-1447275400 tempest-TestMinimumBasicScenario-1447275400-project] [instance: 401ec9c3-4ff3-468a-b334-606c558b8d7a] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:43:18.859517 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-620c7f9e-cdf2-4317-b586-0fb347f9104f tempest-TestMinimumBasicScenario-1447275400 tempest-TestMinimumBasicScenario-1447275400-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:43:18.860184 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-620c7f9e-cdf2-4317-b586-0fb347f9104f tempest-TestMinimumBasicScenario-1447275400 tempest-TestMinimumBasicScenario-1447275400-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:43:18.860836 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-620c7f9e-cdf2-4317-b586-0fb347f9104f tempest-TestMinimumBasicScenario-1447275400 tempest-TestMinimumBasicScenario-1447275400-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:43:18.861925 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-620c7f9e-cdf2-4317-b586-0fb347f9104f tempest-TestMinimumBasicScenario-1447275400 tempest-TestMinimumBasicScenario-1447275400-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:43:18.862556 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-620c7f9e-cdf2-4317-b586-0fb347f9104f tempest-TestMinimumBasicScenario-1447275400 tempest-TestMinimumBasicScenario-1447275400-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:43:18.863003 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-620c7f9e-cdf2-4317-b586-0fb347f9104f tempest-TestMinimumBasicScenario-1447275400 tempest-TestMinimumBasicScenario-1447275400-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:43:19.007807 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-620c7f9e-cdf2-4317-b586-0fb347f9104f tempest-TestMinimumBasicScenario-1447275400 tempest-TestMinimumBasicScenario-1447275400-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ba73716667184588b522bc7856404be8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7dd5f20c4a714ee3b555d4aef8b87730', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:43:19.016227 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-620c7f9e-cdf2-4317-b586-0fb347f9104f tempest-TestMinimumBasicScenario-1447275400 tempest-TestMinimumBasicScenario-1447275400-project] 10.222.0.52 "GET /compute/v2.1/servers/401ec9c3-4ff3-468a-b334-606c558b8d7a" status: 200 len: 1547 microversion: 2.1 time: 0.173742 Aug 27 02:43:19.016794 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 612/1230] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:43:18 2022] GET /compute/v2.1/servers/401ec9c3-4ff3-468a-b334-606c558b8d7a => generated 1547 bytes in 175 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 27 02:43:19.036967 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-81093918-f716-4495-bff3-9a1c33b0fd7a tempest-TestMinimumBasicScenario-1447275400 tempest-TestMinimumBasicScenario-1447275400-project] Calling method '>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:43:19.037597 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-81093918-f716-4495-bff3-9a1c33b0fd7a tempest-TestMinimumBasicScenario-1447275400 tempest-TestMinimumBasicScenario-1447275400-project] [instance: 401ec9c3-4ff3-468a-b334-606c558b8d7a] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:43:19.044447 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-81093918-f716-4495-bff3-9a1c33b0fd7a tempest-TestMinimumBasicScenario-1447275400 tempest-TestMinimumBasicScenario-1447275400-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:43:19.044925 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-81093918-f716-4495-bff3-9a1c33b0fd7a tempest-TestMinimumBasicScenario-1447275400 tempest-TestMinimumBasicScenario-1447275400-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:43:19.045385 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-81093918-f716-4495-bff3-9a1c33b0fd7a tempest-TestMinimumBasicScenario-1447275400 tempest-TestMinimumBasicScenario-1447275400-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:43:19.046063 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-81093918-f716-4495-bff3-9a1c33b0fd7a tempest-TestMinimumBasicScenario-1447275400 tempest-TestMinimumBasicScenario-1447275400-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:43:19.046508 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-81093918-f716-4495-bff3-9a1c33b0fd7a tempest-TestMinimumBasicScenario-1447275400 tempest-TestMinimumBasicScenario-1447275400-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:43:19.046921 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-81093918-f716-4495-bff3-9a1c33b0fd7a tempest-TestMinimumBasicScenario-1447275400 tempest-TestMinimumBasicScenario-1447275400-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:43:19.188122 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-81093918-f716-4495-bff3-9a1c33b0fd7a tempest-TestMinimumBasicScenario-1447275400 tempest-TestMinimumBasicScenario-1447275400-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ba73716667184588b522bc7856404be8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7dd5f20c4a714ee3b555d4aef8b87730', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:43:19.196627 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-81093918-f716-4495-bff3-9a1c33b0fd7a tempest-TestMinimumBasicScenario-1447275400 tempest-TestMinimumBasicScenario-1447275400-project] 10.222.0.52 "GET /compute/v2.1/servers/401ec9c3-4ff3-468a-b334-606c558b8d7a" status: 200 len: 1547 microversion: 2.1 time: 0.163537 Aug 27 02:43:19.197216 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 619/1231] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:43:19 2022] GET /compute/v2.1/servers/401ec9c3-4ff3-468a-b334-606c558b8d7a => generated 1547 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 27 02:43:19.216937 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-031def94-1fcb-4405-92d0-4638f8985ef3 tempest-TestMinimumBasicScenario-1447275400 tempest-TestMinimumBasicScenario-1447275400-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:43:19.222090 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-031def94-1fcb-4405-92d0-4638f8985ef3 tempest-TestMinimumBasicScenario-1447275400 tempest-TestMinimumBasicScenario-1447275400-project] Policy check for os_compute_api:servers:allow_all_filters failed with credentials {'is_admin': False, 'user_id': 'ba73716667184588b522bc7856404be8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7dd5f20c4a714ee3b555d4aef8b87730', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:43:19.222847 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-031def94-1fcb-4405-92d0-4638f8985ef3 tempest-TestMinimumBasicScenario-1447275400 tempest-TestMinimumBasicScenario-1447275400-project] Searching by: {'deleted': False, 'project_id': '7dd5f20c4a714ee3b555d4aef8b87730'} {{(pid=120863) get_all /opt/stack/nova/nova/compute/api.py:3017}} Aug 27 02:43:19.229538 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-031def94-1fcb-4405-92d0-4638f8985ef3 tempest-TestMinimumBasicScenario-1447275400 tempest-TestMinimumBasicScenario-1447275400-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:43:19.230008 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-031def94-1fcb-4405-92d0-4638f8985ef3 tempest-TestMinimumBasicScenario-1447275400 tempest-TestMinimumBasicScenario-1447275400-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:43:19.230433 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-031def94-1fcb-4405-92d0-4638f8985ef3 tempest-TestMinimumBasicScenario-1447275400 tempest-TestMinimumBasicScenario-1447275400-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:43:19.231140 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-031def94-1fcb-4405-92d0-4638f8985ef3 tempest-TestMinimumBasicScenario-1447275400 tempest-TestMinimumBasicScenario-1447275400-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:43:19.231588 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-031def94-1fcb-4405-92d0-4638f8985ef3 tempest-TestMinimumBasicScenario-1447275400 tempest-TestMinimumBasicScenario-1447275400-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:43:19.232003 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-031def94-1fcb-4405-92d0-4638f8985ef3 tempest-TestMinimumBasicScenario-1447275400 tempest-TestMinimumBasicScenario-1447275400-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:43:19.325577 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.multi_cell_list [None req-031def94-1fcb-4405-92d0-4638f8985ef3 tempest-TestMinimumBasicScenario-1447275400 tempest-TestMinimumBasicScenario-1447275400-project] Listed batch of 1 results from cell out of 1000 limit. Returned 1 total so far. {{(pid=120863) do_query /opt/stack/nova/nova/compute/multi_cell_list.py:373}} Aug 27 02:43:19.428949 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-031def94-1fcb-4405-92d0-4638f8985ef3 tempest-TestMinimumBasicScenario-1447275400 tempest-TestMinimumBasicScenario-1447275400-project] 10.222.0.52 "GET /compute/v2.1/servers" status: 200 len: 343 microversion: 2.1 time: 0.215709 Aug 27 02:43:19.429543 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 613/1232] 10.222.0.52 () {58 vars in 1125 bytes} [Sat Aug 27 02:43:19 2022] GET /compute/v2.1/servers => generated 343 bytes in 217 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 27 02:43:19.441753 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-a3e45108-b066-45b8-8cec-35524bfa6019 tempest-TestMinimumBasicScenario-1447275400 tempest-TestMinimumBasicScenario-1447275400-project] Calling method '>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:43:19.442354 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-a3e45108-b066-45b8-8cec-35524bfa6019 tempest-TestMinimumBasicScenario-1447275400 tempest-TestMinimumBasicScenario-1447275400-project] [instance: 401ec9c3-4ff3-468a-b334-606c558b8d7a] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:43:19.448366 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-a3e45108-b066-45b8-8cec-35524bfa6019 tempest-TestMinimumBasicScenario-1447275400 tempest-TestMinimumBasicScenario-1447275400-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:43:19.448875 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-a3e45108-b066-45b8-8cec-35524bfa6019 tempest-TestMinimumBasicScenario-1447275400 tempest-TestMinimumBasicScenario-1447275400-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:43:19.449355 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-a3e45108-b066-45b8-8cec-35524bfa6019 tempest-TestMinimumBasicScenario-1447275400 tempest-TestMinimumBasicScenario-1447275400-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:43:19.450047 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-a3e45108-b066-45b8-8cec-35524bfa6019 tempest-TestMinimumBasicScenario-1447275400 tempest-TestMinimumBasicScenario-1447275400-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:43:19.450490 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-a3e45108-b066-45b8-8cec-35524bfa6019 tempest-TestMinimumBasicScenario-1447275400 tempest-TestMinimumBasicScenario-1447275400-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:43:19.450907 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-a3e45108-b066-45b8-8cec-35524bfa6019 tempest-TestMinimumBasicScenario-1447275400 tempest-TestMinimumBasicScenario-1447275400-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:43:19.600675 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-a3e45108-b066-45b8-8cec-35524bfa6019 tempest-TestMinimumBasicScenario-1447275400 tempest-TestMinimumBasicScenario-1447275400-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ba73716667184588b522bc7856404be8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7dd5f20c4a714ee3b555d4aef8b87730', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:43:19.612176 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-a3e45108-b066-45b8-8cec-35524bfa6019 tempest-TestMinimumBasicScenario-1447275400 tempest-TestMinimumBasicScenario-1447275400-project] 10.222.0.52 "GET /compute/v2.1/servers/401ec9c3-4ff3-468a-b334-606c558b8d7a" status: 200 len: 1547 microversion: 2.1 time: 0.174027 Aug 27 02:43:19.612836 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 620/1233] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:43:19 2022] GET /compute/v2.1/servers/401ec9c3-4ff3-468a-b334-606c558b8d7a => generated 1547 bytes in 176 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:43:21.174939 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-eb150bdd-3463-4e62-b271-4da6406d48ba tempest-TestMinimumBasicScenario-1447275400 tempest-TestMinimumBasicScenario-1447275400-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "f3c255b5-9631-420b-ad9d-b251e58e39ae"}} {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 27 02:43:21.177326 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-eb150bdd-3463-4e62-b271-4da6406d48ba tempest-TestMinimumBasicScenario-1447275400 tempest-TestMinimumBasicScenario-1447275400-project] [instance: 401ec9c3-4ff3-468a-b334-606c558b8d7a] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:43:21.184647 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-eb150bdd-3463-4e62-b271-4da6406d48ba tempest-TestMinimumBasicScenario-1447275400 tempest-TestMinimumBasicScenario-1447275400-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:43:21.185145 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-eb150bdd-3463-4e62-b271-4da6406d48ba tempest-TestMinimumBasicScenario-1447275400 tempest-TestMinimumBasicScenario-1447275400-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:43:21.185616 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-eb150bdd-3463-4e62-b271-4da6406d48ba tempest-TestMinimumBasicScenario-1447275400 tempest-TestMinimumBasicScenario-1447275400-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:43:21.186317 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-eb150bdd-3463-4e62-b271-4da6406d48ba tempest-TestMinimumBasicScenario-1447275400 tempest-TestMinimumBasicScenario-1447275400-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:43:21.186769 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-eb150bdd-3463-4e62-b271-4da6406d48ba tempest-TestMinimumBasicScenario-1447275400 tempest-TestMinimumBasicScenario-1447275400-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:43:21.187196 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-eb150bdd-3463-4e62-b271-4da6406d48ba tempest-TestMinimumBasicScenario-1447275400 tempest-TestMinimumBasicScenario-1447275400-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:43:21.480419 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-eb150bdd-3463-4e62-b271-4da6406d48ba tempest-TestMinimumBasicScenario-1447275400 tempest-TestMinimumBasicScenario-1447275400-project] 10.222.0.52 "POST /compute/v2.1/servers/401ec9c3-4ff3-468a-b334-606c558b8d7a/os-volume_attachments" status: 200 len: 194 microversion: 2.1 time: 0.309048 Aug 27 02:43:21.480955 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 614/1234] 10.222.0.52 () {60 vars in 1323 bytes} [Sat Aug 27 02:43:21 2022] POST /compute/v2.1/servers/401ec9c3-4ff3-468a-b334-606c558b8d7a/os-volume_attachments => generated 194 bytes in 311 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 27 02:43:29.132733 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-c297a9f6-db64-461b-baa4-fe7aa33fedd2 tempest-TestMinimumBasicScenario-1447275400 tempest-TestMinimumBasicScenario-1447275400-project] Calling method '>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:43:29.133614 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-c297a9f6-db64-461b-baa4-fe7aa33fedd2 tempest-TestMinimumBasicScenario-1447275400 tempest-TestMinimumBasicScenario-1447275400-project] [instance: 401ec9c3-4ff3-468a-b334-606c558b8d7a] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:43:29.136088 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG dbcounter [-] [120864] Writing DB stats nova_api:SELECT=40,nova_api:INSERT=3 {{(pid=120864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 27 02:43:29.141535 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-c297a9f6-db64-461b-baa4-fe7aa33fedd2 tempest-TestMinimumBasicScenario-1447275400 tempest-TestMinimumBasicScenario-1447275400-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:43:29.142056 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-c297a9f6-db64-461b-baa4-fe7aa33fedd2 tempest-TestMinimumBasicScenario-1447275400 tempest-TestMinimumBasicScenario-1447275400-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:43:29.142676 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-c297a9f6-db64-461b-baa4-fe7aa33fedd2 tempest-TestMinimumBasicScenario-1447275400 tempest-TestMinimumBasicScenario-1447275400-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:43:29.143515 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-c297a9f6-db64-461b-baa4-fe7aa33fedd2 tempest-TestMinimumBasicScenario-1447275400 tempest-TestMinimumBasicScenario-1447275400-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:43:29.143989 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-c297a9f6-db64-461b-baa4-fe7aa33fedd2 tempest-TestMinimumBasicScenario-1447275400 tempest-TestMinimumBasicScenario-1447275400-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:43:29.144409 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-c297a9f6-db64-461b-baa4-fe7aa33fedd2 tempest-TestMinimumBasicScenario-1447275400 tempest-TestMinimumBasicScenario-1447275400-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:43:29.147598 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG dbcounter [-] [120864] Writing DB stats nova_cell1:SELECT=75 {{(pid=120864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 27 02:43:29.276984 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-c297a9f6-db64-461b-baa4-fe7aa33fedd2 tempest-TestMinimumBasicScenario-1447275400 tempest-TestMinimumBasicScenario-1447275400-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ba73716667184588b522bc7856404be8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7dd5f20c4a714ee3b555d4aef8b87730', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:43:29.286263 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-c297a9f6-db64-461b-baa4-fe7aa33fedd2 tempest-TestMinimumBasicScenario-1447275400 tempest-TestMinimumBasicScenario-1447275400-project] 10.222.0.52 "GET /compute/v2.1/servers/401ec9c3-4ff3-468a-b334-606c558b8d7a" status: 200 len: 1593 microversion: 2.1 time: 0.157392 Aug 27 02:43:29.286849 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 621/1235] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:43:29 2022] GET /compute/v2.1/servers/401ec9c3-4ff3-468a-b334-606c558b8d7a => generated 1593 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:43:31.425760 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG dbcounter [-] [120863] Writing DB stats nova_cell0:SELECT=2 {{(pid=120863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 27 02:43:31.427041 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG dbcounter [-] [120863] Writing DB stats nova_api:SELECT=39,nova_api:INSERT=1 {{(pid=120863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 27 02:43:31.470399 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG dbcounter [-] [120863] Writing DB stats nova_cell1:SELECT=71,nova_cell1:UPDATE=2,nova_cell1:INSERT=1 {{(pid=120863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 27 02:43:31.619147 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [req-2d351f27-e56d-4d65-a989-6bc9221513d5 req-51306c72-319c-44aa-916d-017a4ef0b9d0 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "401ec9c3-4ff3-468a-b334-606c558b8d7a", "tag": "974dc3f6-d8bf-4ab5-a284-a4ecb61b8866"}]} {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 27 02:43:31.631112 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [req-2d351f27-e56d-4d65-a989-6bc9221513d5 req-51306c72-319c-44aa-916d-017a4ef0b9d0 service nova] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:43:31.631590 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [req-2d351f27-e56d-4d65-a989-6bc9221513d5 req-51306c72-319c-44aa-916d-017a4ef0b9d0 service nova] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:43:31.632121 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [req-2d351f27-e56d-4d65-a989-6bc9221513d5 req-51306c72-319c-44aa-916d-017a4ef0b9d0 service nova] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:43:31.644245 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.compute.server_external_events [req-2d351f27-e56d-4d65-a989-6bc9221513d5 req-51306c72-319c-44aa-916d-017a4ef0b9d0 service nova] Creating event network-changed:974dc3f6-d8bf-4ab5-a284-a4ecb61b8866 for instance 401ec9c3-4ff3-468a-b334-606c558b8d7a on hv-cinder-84630 Aug 27 02:43:31.649569 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [req-2d351f27-e56d-4d65-a989-6bc9221513d5 req-51306c72-319c-44aa-916d-017a4ef0b9d0 service nova] 10.222.0.52 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.230982 Aug 27 02:43:31.650600 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 615/1236] 10.222.0.52 () {64 vars in 1317 bytes} [Sat Aug 27 02:43:31 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 233 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 27 02:43:32.594473 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-70ac4e03-8518-4f3b-a364-3c846ef3e9ed tempest-TestMinimumBasicScenario-1447275400 tempest-TestMinimumBasicScenario-1447275400-project] Action: 'action', calling method: >, body: {"addSecurityGroup": {"name": "tempest-secgroup-smoke-1539641839"}} {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 27 02:43:32.595099 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-70ac4e03-8518-4f3b-a364-3c846ef3e9ed tempest-TestMinimumBasicScenario-1447275400 tempest-TestMinimumBasicScenario-1447275400-project] [instance: 401ec9c3-4ff3-468a-b334-606c558b8d7a] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:43:32.601873 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-70ac4e03-8518-4f3b-a364-3c846ef3e9ed tempest-TestMinimumBasicScenario-1447275400 tempest-TestMinimumBasicScenario-1447275400-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:43:32.602423 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-70ac4e03-8518-4f3b-a364-3c846ef3e9ed tempest-TestMinimumBasicScenario-1447275400 tempest-TestMinimumBasicScenario-1447275400-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:43:32.603037 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-70ac4e03-8518-4f3b-a364-3c846ef3e9ed tempest-TestMinimumBasicScenario-1447275400 tempest-TestMinimumBasicScenario-1447275400-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:43:32.603720 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-70ac4e03-8518-4f3b-a364-3c846ef3e9ed tempest-TestMinimumBasicScenario-1447275400 tempest-TestMinimumBasicScenario-1447275400-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:43:32.604162 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-70ac4e03-8518-4f3b-a364-3c846ef3e9ed tempest-TestMinimumBasicScenario-1447275400 tempest-TestMinimumBasicScenario-1447275400-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:43:32.604604 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-70ac4e03-8518-4f3b-a364-3c846ef3e9ed tempest-TestMinimumBasicScenario-1447275400 tempest-TestMinimumBasicScenario-1447275400-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:43:32.741334 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.network.security_group_api [None req-70ac4e03-8518-4f3b-a364-3c846ef3e9ed tempest-TestMinimumBasicScenario-1447275400 tempest-TestMinimumBasicScenario-1447275400-project] Adding security group a4941718-8a89-4358-b890-d2d55433ecb9 to port 974dc3f6-d8bf-4ab5-a284-a4ecb61b8866 Aug 27 02:43:33.088208 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-70ac4e03-8518-4f3b-a364-3c846ef3e9ed tempest-TestMinimumBasicScenario-1447275400 tempest-TestMinimumBasicScenario-1447275400-project] 10.222.0.52 "POST /compute/v2.1/servers/401ec9c3-4ff3-468a-b334-606c558b8d7a/action" status: 202 len: 0 microversion: 2.1 time: 0.497695 Aug 27 02:43:33.088851 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 622/1237] 10.222.0.52 () {60 vars in 1278 bytes} [Sat Aug 27 02:43:32 2022] POST /compute/v2.1/servers/401ec9c3-4ff3-468a-b334-606c558b8d7a/action => generated 0 bytes in 499 msecs (HTTP/1.1 202) 9 headers in 361 bytes (2 switches on core 0) Aug 27 02:43:33.096657 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-1c6de75d-66fb-4163-a87f-0088f9f4e9d0 tempest-TestMinimumBasicScenario-1447275400 tempest-TestMinimumBasicScenario-1447275400-project] Calling method '>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:43:33.097301 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-1c6de75d-66fb-4163-a87f-0088f9f4e9d0 tempest-TestMinimumBasicScenario-1447275400 tempest-TestMinimumBasicScenario-1447275400-project] [instance: 401ec9c3-4ff3-468a-b334-606c558b8d7a] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:43:33.097434 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [req-bf0bc325-4a77-422c-ac19-4cdccba1ba58 req-fcf58d15-0147-45e4-b277-da86382df8b7 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "401ec9c3-4ff3-468a-b334-606c558b8d7a", "tag": "974dc3f6-d8bf-4ab5-a284-a4ecb61b8866"}]} {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 27 02:43:33.103695 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-1c6de75d-66fb-4163-a87f-0088f9f4e9d0 tempest-TestMinimumBasicScenario-1447275400 tempest-TestMinimumBasicScenario-1447275400-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:43:33.104245 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-1c6de75d-66fb-4163-a87f-0088f9f4e9d0 tempest-TestMinimumBasicScenario-1447275400 tempest-TestMinimumBasicScenario-1447275400-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:43:33.104691 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-1c6de75d-66fb-4163-a87f-0088f9f4e9d0 tempest-TestMinimumBasicScenario-1447275400 tempest-TestMinimumBasicScenario-1447275400-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:43:33.105447 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-1c6de75d-66fb-4163-a87f-0088f9f4e9d0 tempest-TestMinimumBasicScenario-1447275400 tempest-TestMinimumBasicScenario-1447275400-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:43:33.105928 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-1c6de75d-66fb-4163-a87f-0088f9f4e9d0 tempest-TestMinimumBasicScenario-1447275400 tempest-TestMinimumBasicScenario-1447275400-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:43:33.106419 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-1c6de75d-66fb-4163-a87f-0088f9f4e9d0 tempest-TestMinimumBasicScenario-1447275400 tempest-TestMinimumBasicScenario-1447275400-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:43:33.108104 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [req-bf0bc325-4a77-422c-ac19-4cdccba1ba58 req-fcf58d15-0147-45e4-b277-da86382df8b7 service nova] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:43:33.108719 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [req-bf0bc325-4a77-422c-ac19-4cdccba1ba58 req-fcf58d15-0147-45e4-b277-da86382df8b7 service nova] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:43:33.109217 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [req-bf0bc325-4a77-422c-ac19-4cdccba1ba58 req-fcf58d15-0147-45e4-b277-da86382df8b7 service nova] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:43:33.126264 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.compute.server_external_events [req-bf0bc325-4a77-422c-ac19-4cdccba1ba58 req-fcf58d15-0147-45e4-b277-da86382df8b7 service nova] Creating event network-changed:974dc3f6-d8bf-4ab5-a284-a4ecb61b8866 for instance 401ec9c3-4ff3-468a-b334-606c558b8d7a on hv-cinder-84630 Aug 27 02:43:33.132052 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [req-bf0bc325-4a77-422c-ac19-4cdccba1ba58 req-fcf58d15-0147-45e4-b277-da86382df8b7 service nova] 10.222.0.52 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.039230 Aug 27 02:43:33.133006 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 616/1238] 10.222.0.52 () {64 vars in 1317 bytes} [Sat Aug 27 02:43:33 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 41 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Aug 27 02:43:33.305500 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-1c6de75d-66fb-4163-a87f-0088f9f4e9d0 tempest-TestMinimumBasicScenario-1447275400 tempest-TestMinimumBasicScenario-1447275400-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ba73716667184588b522bc7856404be8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7dd5f20c4a714ee3b555d4aef8b87730', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:43:33.314990 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-1c6de75d-66fb-4163-a87f-0088f9f4e9d0 tempest-TestMinimumBasicScenario-1447275400 tempest-TestMinimumBasicScenario-1447275400-project] 10.222.0.52 "GET /compute/v2.1/servers/401ec9c3-4ff3-468a-b334-606c558b8d7a" status: 200 len: 1759 microversion: 2.1 time: 0.221727 Aug 27 02:43:33.315811 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 623/1239] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:43:33 2022] GET /compute/v2.1/servers/401ec9c3-4ff3-468a-b334-606c558b8d7a => generated 1759 bytes in 223 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:43:49.644725 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [req-1095e796-27d6-4cf4-aaf4-8d4c2949bc0a req-2a537680-76e4-4eb8-a2a1-deb0153a9537 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "a29b4020-7f17-480c-b0f4-062c82eaddd2", "tag": "0f6ab26e-c941-4e20-b27e-ed8630548227"}]} {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 27 02:43:49.645778 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG dbcounter [-] [120863] Writing DB stats nova_api:SELECT=4 {{(pid=120863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 27 02:43:49.646400 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG dbcounter [-] [120863] Writing DB stats nova_cell1:SELECT=4 {{(pid=120863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 27 02:43:49.655011 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG dbcounter [-] [120864] Writing DB stats nova_api:SELECT=5 {{(pid=120864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 27 02:43:49.656352 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG dbcounter [-] [120864] Writing DB stats nova_cell1:SELECT=15 {{(pid=120864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 27 02:43:49.661222 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [req-1095e796-27d6-4cf4-aaf4-8d4c2949bc0a req-2a537680-76e4-4eb8-a2a1-deb0153a9537 service nova] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:43:49.664368 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-c1e43ef0-b920-4493-bd6c-4363e67e3477 tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:43:49.664973 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-c1e43ef0-b920-4493-bd6c-4363e67e3477 tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] [instance: a29b4020-7f17-480c-b0f4-062c82eaddd2] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:43:49.672498 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-c1e43ef0-b920-4493-bd6c-4363e67e3477 tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:43:49.673726 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-c1e43ef0-b920-4493-bd6c-4363e67e3477 tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:43:49.674261 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-c1e43ef0-b920-4493-bd6c-4363e67e3477 tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:43:49.674981 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-c1e43ef0-b920-4493-bd6c-4363e67e3477 tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:43:49.675464 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-c1e43ef0-b920-4493-bd6c-4363e67e3477 tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:43:49.675857 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-c1e43ef0-b920-4493-bd6c-4363e67e3477 tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:43:49.677853 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [req-1095e796-27d6-4cf4-aaf4-8d4c2949bc0a req-2a537680-76e4-4eb8-a2a1-deb0153a9537 service nova] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.017s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:43:49.678287 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [req-1095e796-27d6-4cf4-aaf4-8d4c2949bc0a req-2a537680-76e4-4eb8-a2a1-deb0153a9537 service nova] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:43:49.694256 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.compute.server_external_events [req-1095e796-27d6-4cf4-aaf4-8d4c2949bc0a req-2a537680-76e4-4eb8-a2a1-deb0153a9537 service nova] Creating event network-changed:0f6ab26e-c941-4e20-b27e-ed8630548227 for instance a29b4020-7f17-480c-b0f4-062c82eaddd2 on hv-cinder-84630 Aug 27 02:43:49.704095 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [req-1095e796-27d6-4cf4-aaf4-8d4c2949bc0a req-2a537680-76e4-4eb8-a2a1-deb0153a9537 service nova] 10.222.0.52 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.064171 Aug 27 02:43:49.705088 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 624/1240] 10.222.0.52 () {64 vars in 1317 bytes} [Sat Aug 27 02:43:49 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 66 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Aug 27 02:43:49.711265 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-c1e43ef0-b920-4493-bd6c-4363e67e3477 tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] [instance: a29b4020-7f17-480c-b0f4-062c82eaddd2] Going to try to terminate instance {{(pid=120863) delete /opt/stack/nova/nova/compute/api.py:2719}} Aug 27 02:43:49.817362 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-c1e43ef0-b920-4493-bd6c-4363e67e3477 tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] 10.222.0.52 "DELETE /compute/v2.1/servers/a29b4020-7f17-480c-b0f4-062c82eaddd2" status: 204 len: 0 microversion: 2.1 time: 0.180341 Aug 27 02:43:49.818162 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 617/1241] 10.222.0.52 () {58 vars in 1239 bytes} [Sat Aug 27 02:43:49 2022] DELETE /compute/v2.1/servers/a29b4020-7f17-480c-b0f4-062c82eaddd2 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Aug 27 02:43:49.827173 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-f3ebff30-45ef-4878-9401-5798d3263dc2 tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Calling method '>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:43:49.827878 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-f3ebff30-45ef-4878-9401-5798d3263dc2 tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] [instance: a29b4020-7f17-480c-b0f4-062c82eaddd2] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:43:49.834924 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-f3ebff30-45ef-4878-9401-5798d3263dc2 tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:43:49.835579 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-f3ebff30-45ef-4878-9401-5798d3263dc2 tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:43:49.836023 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-f3ebff30-45ef-4878-9401-5798d3263dc2 tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:43:49.836797 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-f3ebff30-45ef-4878-9401-5798d3263dc2 tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:43:49.837216 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-f3ebff30-45ef-4878-9401-5798d3263dc2 tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:43:49.837643 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-f3ebff30-45ef-4878-9401-5798d3263dc2 tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:43:50.039568 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-f3ebff30-45ef-4878-9401-5798d3263dc2 tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3fe5a1a037fa460eb3121e4d307542da', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '79e671941c234f36b542080d256de2c6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:43:50.047735 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-f3ebff30-45ef-4878-9401-5798d3263dc2 tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] 10.222.0.52 "GET /compute/v2.1/servers/a29b4020-7f17-480c-b0f4-062c82eaddd2" status: 200 len: 1577 microversion: 2.1 time: 0.224818 Aug 27 02:43:50.048738 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 625/1242] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:43:49 2022] GET /compute/v2.1/servers/a29b4020-7f17-480c-b0f4-062c82eaddd2 => generated 1577 bytes in 227 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:43:51.071046 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-59adcaea-5d94-4363-a517-d49ac089a4d3 tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:43:51.072464 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-59adcaea-5d94-4363-a517-d49ac089a4d3 tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] [instance: a29b4020-7f17-480c-b0f4-062c82eaddd2] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:43:51.080441 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-59adcaea-5d94-4363-a517-d49ac089a4d3 tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:43:51.080925 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-59adcaea-5d94-4363-a517-d49ac089a4d3 tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:43:51.081415 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-59adcaea-5d94-4363-a517-d49ac089a4d3 tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:43:51.082129 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-59adcaea-5d94-4363-a517-d49ac089a4d3 tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:43:51.082582 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-59adcaea-5d94-4363-a517-d49ac089a4d3 tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:43:51.083016 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-59adcaea-5d94-4363-a517-d49ac089a4d3 tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:43:51.335171 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-59adcaea-5d94-4363-a517-d49ac089a4d3 tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3fe5a1a037fa460eb3121e4d307542da', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '79e671941c234f36b542080d256de2c6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:43:51.343672 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-59adcaea-5d94-4363-a517-d49ac089a4d3 tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] 10.222.0.52 "GET /compute/v2.1/servers/a29b4020-7f17-480c-b0f4-062c82eaddd2" status: 200 len: 1577 microversion: 2.1 time: 0.277284 Aug 27 02:43:51.344641 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 618/1243] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:43:51 2022] GET /compute/v2.1/servers/a29b4020-7f17-480c-b0f4-062c82eaddd2 => generated 1577 bytes in 279 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:43:52.232362 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [req-ed39bb1a-8a61-4f35-b2d9-b96a6e85268d req-d2d734a0-4a69-4feb-9a50-956d55efc374 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "a29b4020-7f17-480c-b0f4-062c82eaddd2", "name": "network-vif-deleted", "tag": "0f6ab26e-c941-4e20-b27e-ed8630548227"}]} {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 27 02:43:52.242674 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [req-ed39bb1a-8a61-4f35-b2d9-b96a6e85268d req-d2d734a0-4a69-4feb-9a50-956d55efc374 service nova] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:43:52.243062 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [req-ed39bb1a-8a61-4f35-b2d9-b96a6e85268d req-d2d734a0-4a69-4feb-9a50-956d55efc374 service nova] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:43:52.243482 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [req-ed39bb1a-8a61-4f35-b2d9-b96a6e85268d req-d2d734a0-4a69-4feb-9a50-956d55efc374 service nova] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:43:52.255470 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.compute.server_external_events [req-ed39bb1a-8a61-4f35-b2d9-b96a6e85268d req-d2d734a0-4a69-4feb-9a50-956d55efc374 service nova] Creating event network-vif-deleted:0f6ab26e-c941-4e20-b27e-ed8630548227 for instance a29b4020-7f17-480c-b0f4-062c82eaddd2 on hv-cinder-84630 Aug 27 02:43:52.260443 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [req-ed39bb1a-8a61-4f35-b2d9-b96a6e85268d req-d2d734a0-4a69-4feb-9a50-956d55efc374 service nova] 10.222.0.52 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.031736 Aug 27 02:43:52.261027 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 626/1244] 10.222.0.52 () {64 vars in 1317 bytes} [Sat Aug 27 02:43:52 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 33 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 27 02:43:52.366755 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-68edd4a0-87c0-4405-a09f-64f6d03df6c1 tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:43:52.367585 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-68edd4a0-87c0-4405-a09f-64f6d03df6c1 tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] [instance: a29b4020-7f17-480c-b0f4-062c82eaddd2] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:43:52.374224 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-68edd4a0-87c0-4405-a09f-64f6d03df6c1 tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:43:52.374713 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-68edd4a0-87c0-4405-a09f-64f6d03df6c1 tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:43:52.375151 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-68edd4a0-87c0-4405-a09f-64f6d03df6c1 tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:43:52.375878 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-68edd4a0-87c0-4405-a09f-64f6d03df6c1 tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:43:52.376580 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-68edd4a0-87c0-4405-a09f-64f6d03df6c1 tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:43:52.376952 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-68edd4a0-87c0-4405-a09f-64f6d03df6c1 tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:43:52.448893 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-68edd4a0-87c0-4405-a09f-64f6d03df6c1 tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3fe5a1a037fa460eb3121e4d307542da', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '79e671941c234f36b542080d256de2c6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:43:52.458705 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-68edd4a0-87c0-4405-a09f-64f6d03df6c1 tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] 10.222.0.52 "GET /compute/v2.1/servers/a29b4020-7f17-480c-b0f4-062c82eaddd2" status: 200 len: 1226 microversion: 2.1 time: 0.096838 Aug 27 02:43:52.459275 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 619/1245] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:43:52 2022] GET /compute/v2.1/servers/a29b4020-7f17-480c-b0f4-062c82eaddd2 => generated 1226 bytes in 98 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:43:53.486168 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-d1e2faaf-a24e-4693-899c-b679f044856f tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Calling method '>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:43:53.487011 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-d1e2faaf-a24e-4693-899c-b679f044856f tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] [instance: a29b4020-7f17-480c-b0f4-062c82eaddd2] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:43:53.493351 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-d1e2faaf-a24e-4693-899c-b679f044856f tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:43:53.493857 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-d1e2faaf-a24e-4693-899c-b679f044856f tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:43:53.494289 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-d1e2faaf-a24e-4693-899c-b679f044856f tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:43:53.495294 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-d1e2faaf-a24e-4693-899c-b679f044856f tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:43:53.495744 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-d1e2faaf-a24e-4693-899c-b679f044856f tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:43:53.496166 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-d1e2faaf-a24e-4693-899c-b679f044856f tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:43:53.512010 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.wsgi [None req-d1e2faaf-a24e-4693-899c-b679f044856f tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] HTTP exception thrown: Instance a29b4020-7f17-480c-b0f4-062c82eaddd2 could not be found. Aug 27 02:43:53.513688 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-d1e2faaf-a24e-4693-899c-b679f044856f tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Request method failure captured: Aug 27 02:43:53.513688 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: request: GET /compute/v2.1/servers/a29b4020-7f17-480c-b0f4-062c82eaddd2 HTTP/1.1 Aug 27 02:43:53.513688 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: Accept: application/json Aug 27 02:43:53.513688 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: Accept-Encoding: identity Aug 27 02:43:53.513688 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: Connection: close Aug 27 02:43:53.513688 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: Content-Length: 0 Aug 27 02:43:53.513688 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: Content-Type: application/json Aug 27 02:43:53.513688 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: Host: 10.222.0.52 Aug 27 02:43:53.513688 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: Openstack-System-Scope: None Aug 27 02:43:53.513688 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: User-Agent: python-urllib3/1.26.11 Aug 27 02:43:53.513688 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-Auth-Token: gAAAAABjCYLy85Uu3BiyLepTvyHrkionXI7siD38Rns3gISZZwndflckLvltMlTBHZJ6gk3GKZLgp1sOCr8IN2oWQwuE2aDeO0w7-eDMbQMQ90apHhdn2f8pHdjAco2Aqv2Vx1ThWF0nFT8Ht9XOqMF1ANor3utAF0ObK7gJJB5bN6WCMw2HLSw Aug 27 02:43:53.513688 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-Domain-Id: None Aug 27 02:43:53.513688 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-Domain-Name: None Aug 27 02:43:53.513688 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-Identity-Status: Confirmed Aug 27 02:43:53.513688 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-Is-Admin-Project: True Aug 27 02:43:53.513688 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-Project-Domain-Id: default Aug 27 02:43:53.513688 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-Project-Domain-Name: Default Aug 27 02:43:53.513688 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-Project-Id: 79e671941c234f36b542080d256de2c6 Aug 27 02:43:53.513688 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-Project-Name: tempest-TestVolumeBootPattern-1006469397 Aug 27 02:43:53.513688 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-Role: reader,member Aug 27 02:43:53.513688 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-Roles: reader,member Aug 27 02:43:53.515626 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-Service-Catalog: [{"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.52:8080", "publicURL": "http://10.222.0.52:8080/v1/AUTH_79e671941c234f36b542080d256de2c6"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52:9696/networking"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/compute/v2/79e671941c234f36b542080d256de2c6"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/image"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/volume/v3/79e671941c234f36b542080d256de2c6"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/identity"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/volume/v3/79e671941c234f36b542080d256de2c6"}]}] Aug 27 02:43:53.515626 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-Tenant: tempest-TestVolumeBootPattern-1006469397 Aug 27 02:43:53.515626 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-Tenant-Id: 79e671941c234f36b542080d256de2c6 Aug 27 02:43:53.515626 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-Tenant-Name: tempest-TestVolumeBootPattern-1006469397 Aug 27 02:43:53.515626 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-User: tempest-TestVolumeBootPattern-1006469397-project Aug 27 02:43:53.515626 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-User-Domain-Id: default Aug 27 02:43:53.515626 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-User-Domain-Name: Default Aug 27 02:43:53.515626 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-User-Id: 3fe5a1a037fa460eb3121e4d307542da Aug 27 02:43:53.515626 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-User-Name: tempest-TestVolumeBootPattern-1006469397-project Aug 27 02:43:53.515626 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: method: > Aug 27 02:43:53.515626 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: exception: Instance a29b4020-7f17-480c-b0f4-062c82eaddd2 could not be found. Aug 27 02:43:53.515626 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: action_args: {'id': 'a29b4020-7f17-480c-b0f4-062c82eaddd2'} Aug 27 02:43:53.517190 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Aug 27 02:43:53.517190 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 27 02:43:53.517190 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Aug 27 02:43:53.517190 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Aug 27 02:43:53.517190 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Aug 27 02:43:53.517190 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Aug 27 02:43:53.517190 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Aug 27 02:43:53.517190 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Aug 27 02:43:53.517190 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Aug 27 02:43:53.517190 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Aug 27 02:43:53.517190 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance a29b4020-7f17-480c-b0f4-062c82eaddd2 could not be found. Aug 27 02:43:53.517190 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi Aug 27 02:43:53.517190 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 27 02:43:53.517190 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi Aug 27 02:43:53.517190 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 27 02:43:53.517190 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 27 02:43:53.517190 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 27 02:43:53.517190 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Aug 27 02:43:53.517190 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Aug 27 02:43:53.517190 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Aug 27 02:43:53.519126 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Aug 27 02:43:53.519126 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Aug 27 02:43:53.519126 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi instance = self._get_server( Aug 27 02:43:53.519126 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Aug 27 02:43:53.519126 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Aug 27 02:43:53.519126 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Aug 27 02:43:53.519126 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Aug 27 02:43:53.519126 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance a29b4020-7f17-480c-b0f4-062c82eaddd2 could not be found. Aug 27 02:43:53.519126 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi Aug 27 02:43:53.519126 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 27 02:43:53.519126 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi Aug 27 02:43:53.519126 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 27 02:43:53.519126 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 27 02:43:53.519126 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 27 02:43:53.519126 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Aug 27 02:43:53.519126 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Aug 27 02:43:53.519126 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance a29b4020-7f17-480c-b0f4-062c82eaddd2 could not be found. Aug 27 02:43:53.519126 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi Aug 27 02:43:53.520966 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-d1e2faaf-a24e-4693-899c-b679f044856f tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Returning 404 to user: Instance a29b4020-7f17-480c-b0f4-062c82eaddd2 could not be found. {{(pid=120864) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Aug 27 02:43:53.520966 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-d1e2faaf-a24e-4693-899c-b679f044856f tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] 10.222.0.52 "GET /compute/v2.1/servers/a29b4020-7f17-480c-b0f4-062c82eaddd2" status: 404 len: 111 microversion: 2.1 time: 0.035918 Aug 27 02:43:53.520966 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 627/1246] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:43:53 2022] GET /compute/v2.1/servers/a29b4020-7f17-480c-b0f4-062c82eaddd2 => generated 111 bytes in 38 msecs (HTTP/1.1 404) 9 headers in 379 bytes (2 switches on core 0) Aug 27 02:43:56.680297 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-7928e526-fa8b-4bc2-a3a9-1a2646f92d1a tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:43:56.680831 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-7928e526-fa8b-4bc2-a3a9-1a2646f92d1a tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] [instance: 2afc317e-fd8d-4e86-89d9-aba4b76d83cb] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:43:56.682057 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [req-6e59b0eb-fb51-4adb-a66e-eb9e7f7ca37b req-84e0f8bb-fc60-49a8-bd9c-96a969c52f4a service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "2afc317e-fd8d-4e86-89d9-aba4b76d83cb", "tag": "df547cda-45a3-4fd8-bd2f-f3b5e1938759"}]} {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 27 02:43:56.691846 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-7928e526-fa8b-4bc2-a3a9-1a2646f92d1a tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:43:56.692405 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-7928e526-fa8b-4bc2-a3a9-1a2646f92d1a tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:43:56.692848 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-7928e526-fa8b-4bc2-a3a9-1a2646f92d1a tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:43:56.692975 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [req-6e59b0eb-fb51-4adb-a66e-eb9e7f7ca37b req-84e0f8bb-fc60-49a8-bd9c-96a969c52f4a service nova] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:43:56.693420 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [req-6e59b0eb-fb51-4adb-a66e-eb9e7f7ca37b req-84e0f8bb-fc60-49a8-bd9c-96a969c52f4a service nova] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:43:56.693583 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-7928e526-fa8b-4bc2-a3a9-1a2646f92d1a tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:43:56.693842 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [req-6e59b0eb-fb51-4adb-a66e-eb9e7f7ca37b req-84e0f8bb-fc60-49a8-bd9c-96a969c52f4a service nova] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:43:56.694250 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-7928e526-fa8b-4bc2-a3a9-1a2646f92d1a tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:43:56.694704 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-7928e526-fa8b-4bc2-a3a9-1a2646f92d1a tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:43:56.706557 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.compute.server_external_events [req-6e59b0eb-fb51-4adb-a66e-eb9e7f7ca37b req-84e0f8bb-fc60-49a8-bd9c-96a969c52f4a service nova] Creating event network-changed:df547cda-45a3-4fd8-bd2f-f3b5e1938759 for instance 2afc317e-fd8d-4e86-89d9-aba4b76d83cb on hv-cinder-84630 Aug 27 02:43:56.711825 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [req-6e59b0eb-fb51-4adb-a66e-eb9e7f7ca37b req-84e0f8bb-fc60-49a8-bd9c-96a969c52f4a service nova] 10.222.0.52 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.033454 Aug 27 02:43:56.713016 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 628/1247] 10.222.0.52 () {64 vars in 1317 bytes} [Sat Aug 27 02:43:56 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 35 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 27 02:43:56.726783 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-7928e526-fa8b-4bc2-a3a9-1a2646f92d1a tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] [instance: 2afc317e-fd8d-4e86-89d9-aba4b76d83cb] Going to try to terminate instance {{(pid=120863) delete /opt/stack/nova/nova/compute/api.py:2719}} Aug 27 02:43:56.817938 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-7928e526-fa8b-4bc2-a3a9-1a2646f92d1a tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] 10.222.0.52 "DELETE /compute/v2.1/servers/2afc317e-fd8d-4e86-89d9-aba4b76d83cb" status: 204 len: 0 microversion: 2.1 time: 0.140822 Aug 27 02:43:56.818737 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 620/1248] 10.222.0.52 () {58 vars in 1239 bytes} [Sat Aug 27 02:43:56 2022] DELETE /compute/v2.1/servers/2afc317e-fd8d-4e86-89d9-aba4b76d83cb => generated 0 bytes in 143 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Aug 27 02:43:56.828196 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-52327648-547b-4a42-89cf-af9fdc4d1d05 tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Calling method '>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:43:56.829739 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-52327648-547b-4a42-89cf-af9fdc4d1d05 tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] [instance: 2afc317e-fd8d-4e86-89d9-aba4b76d83cb] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:43:56.836123 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-52327648-547b-4a42-89cf-af9fdc4d1d05 tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:43:56.836817 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-52327648-547b-4a42-89cf-af9fdc4d1d05 tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:43:56.837594 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-52327648-547b-4a42-89cf-af9fdc4d1d05 tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:43:56.838487 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-52327648-547b-4a42-89cf-af9fdc4d1d05 tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:43:56.839154 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-52327648-547b-4a42-89cf-af9fdc4d1d05 tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:43:56.839606 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-52327648-547b-4a42-89cf-af9fdc4d1d05 tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:43:57.002091 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-52327648-547b-4a42-89cf-af9fdc4d1d05 tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3fe5a1a037fa460eb3121e4d307542da', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '79e671941c234f36b542080d256de2c6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:43:57.010230 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-52327648-547b-4a42-89cf-af9fdc4d1d05 tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] 10.222.0.52 "GET /compute/v2.1/servers/2afc317e-fd8d-4e86-89d9-aba4b76d83cb" status: 200 len: 1579 microversion: 2.1 time: 0.186580 Aug 27 02:43:57.010922 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 629/1249] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:43:56 2022] GET /compute/v2.1/servers/2afc317e-fd8d-4e86-89d9-aba4b76d83cb => generated 1579 bytes in 188 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:43:58.033255 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-70539cb3-2830-47f3-bbca-1f742b96201d tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:43:58.033940 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-70539cb3-2830-47f3-bbca-1f742b96201d tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] [instance: 2afc317e-fd8d-4e86-89d9-aba4b76d83cb] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:43:58.040791 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-70539cb3-2830-47f3-bbca-1f742b96201d tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:43:58.041317 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-70539cb3-2830-47f3-bbca-1f742b96201d tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:43:58.041747 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-70539cb3-2830-47f3-bbca-1f742b96201d tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:43:58.042443 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-70539cb3-2830-47f3-bbca-1f742b96201d tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:43:58.042886 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-70539cb3-2830-47f3-bbca-1f742b96201d tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:43:58.043309 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-70539cb3-2830-47f3-bbca-1f742b96201d tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:43:58.209228 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-70539cb3-2830-47f3-bbca-1f742b96201d tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3fe5a1a037fa460eb3121e4d307542da', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '79e671941c234f36b542080d256de2c6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:43:58.217613 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-70539cb3-2830-47f3-bbca-1f742b96201d tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] 10.222.0.52 "GET /compute/v2.1/servers/2afc317e-fd8d-4e86-89d9-aba4b76d83cb" status: 200 len: 1579 microversion: 2.1 time: 0.188679 Aug 27 02:43:58.218176 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 621/1250] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:43:58 2022] GET /compute/v2.1/servers/2afc317e-fd8d-4e86-89d9-aba4b76d83cb => generated 1579 bytes in 190 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:43:59.187068 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [req-aa004763-104c-4ee0-9006-a7dead987e90 req-bd9886a2-de36-491c-861d-7f3b53c00b29 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "2afc317e-fd8d-4e86-89d9-aba4b76d83cb", "name": "network-vif-deleted", "tag": "df547cda-45a3-4fd8-bd2f-f3b5e1938759"}]} {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 27 02:43:59.197675 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [req-aa004763-104c-4ee0-9006-a7dead987e90 req-bd9886a2-de36-491c-861d-7f3b53c00b29 service nova] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:43:59.198227 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [req-aa004763-104c-4ee0-9006-a7dead987e90 req-bd9886a2-de36-491c-861d-7f3b53c00b29 service nova] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:43:59.198680 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [req-aa004763-104c-4ee0-9006-a7dead987e90 req-bd9886a2-de36-491c-861d-7f3b53c00b29 service nova] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:43:59.211357 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.compute.server_external_events [req-aa004763-104c-4ee0-9006-a7dead987e90 req-bd9886a2-de36-491c-861d-7f3b53c00b29 service nova] Creating event network-vif-deleted:df547cda-45a3-4fd8-bd2f-f3b5e1938759 for instance 2afc317e-fd8d-4e86-89d9-aba4b76d83cb on hv-cinder-84630 Aug 27 02:43:59.216297 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [req-aa004763-104c-4ee0-9006-a7dead987e90 req-bd9886a2-de36-491c-861d-7f3b53c00b29 service nova] 10.222.0.52 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.032921 Aug 27 02:43:59.217316 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 630/1251] 10.222.0.52 () {64 vars in 1317 bytes} [Sat Aug 27 02:43:59 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 35 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 27 02:43:59.240186 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-9e6e19f3-ae40-4e58-861a-0cdd4d8a0b88 tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:43:59.240878 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-9e6e19f3-ae40-4e58-861a-0cdd4d8a0b88 tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] [instance: 2afc317e-fd8d-4e86-89d9-aba4b76d83cb] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:43:59.247724 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-9e6e19f3-ae40-4e58-861a-0cdd4d8a0b88 tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:43:59.248216 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-9e6e19f3-ae40-4e58-861a-0cdd4d8a0b88 tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:43:59.248675 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-9e6e19f3-ae40-4e58-861a-0cdd4d8a0b88 tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:43:59.249433 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-9e6e19f3-ae40-4e58-861a-0cdd4d8a0b88 tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:43:59.250156 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-9e6e19f3-ae40-4e58-861a-0cdd4d8a0b88 tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:43:59.250693 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-9e6e19f3-ae40-4e58-861a-0cdd4d8a0b88 tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:43:59.323011 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-9e6e19f3-ae40-4e58-861a-0cdd4d8a0b88 tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3fe5a1a037fa460eb3121e4d307542da', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '79e671941c234f36b542080d256de2c6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:43:59.331063 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-9e6e19f3-ae40-4e58-861a-0cdd4d8a0b88 tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] 10.222.0.52 "GET /compute/v2.1/servers/2afc317e-fd8d-4e86-89d9-aba4b76d83cb" status: 200 len: 1226 microversion: 2.1 time: 0.094936 Aug 27 02:43:59.331641 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 622/1252] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:43:59 2022] GET /compute/v2.1/servers/2afc317e-fd8d-4e86-89d9-aba4b76d83cb => generated 1226 bytes in 96 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:44:00.370780 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-2379f3a3-fbde-40a4-8547-974706eecd7a tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Calling method '>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:44:00.371870 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-2379f3a3-fbde-40a4-8547-974706eecd7a tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] [instance: 2afc317e-fd8d-4e86-89d9-aba4b76d83cb] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:44:00.380055 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-2379f3a3-fbde-40a4-8547-974706eecd7a tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:44:00.380536 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-2379f3a3-fbde-40a4-8547-974706eecd7a tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:44:00.380965 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-2379f3a3-fbde-40a4-8547-974706eecd7a tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:44:00.381692 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-2379f3a3-fbde-40a4-8547-974706eecd7a tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:44:00.382132 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-2379f3a3-fbde-40a4-8547-974706eecd7a tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:44:00.382592 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-2379f3a3-fbde-40a4-8547-974706eecd7a tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:44:00.397573 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.wsgi [None req-2379f3a3-fbde-40a4-8547-974706eecd7a tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] HTTP exception thrown: Instance 2afc317e-fd8d-4e86-89d9-aba4b76d83cb could not be found. Aug 27 02:44:00.401552 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-2379f3a3-fbde-40a4-8547-974706eecd7a tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Request method failure captured: Aug 27 02:44:00.401552 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: request: GET /compute/v2.1/servers/2afc317e-fd8d-4e86-89d9-aba4b76d83cb HTTP/1.1 Aug 27 02:44:00.401552 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: Accept: application/json Aug 27 02:44:00.401552 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: Accept-Encoding: identity Aug 27 02:44:00.401552 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: Connection: close Aug 27 02:44:00.401552 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: Content-Length: 0 Aug 27 02:44:00.401552 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: Content-Type: application/json Aug 27 02:44:00.401552 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: Host: 10.222.0.52 Aug 27 02:44:00.401552 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: Openstack-System-Scope: None Aug 27 02:44:00.401552 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: User-Agent: python-urllib3/1.26.11 Aug 27 02:44:00.401552 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-Auth-Token: gAAAAABjCYLy85Uu3BiyLepTvyHrkionXI7siD38Rns3gISZZwndflckLvltMlTBHZJ6gk3GKZLgp1sOCr8IN2oWQwuE2aDeO0w7-eDMbQMQ90apHhdn2f8pHdjAco2Aqv2Vx1ThWF0nFT8Ht9XOqMF1ANor3utAF0ObK7gJJB5bN6WCMw2HLSw Aug 27 02:44:00.401552 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-Domain-Id: None Aug 27 02:44:00.401552 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-Domain-Name: None Aug 27 02:44:00.401552 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-Identity-Status: Confirmed Aug 27 02:44:00.401552 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-Is-Admin-Project: True Aug 27 02:44:00.401552 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-Project-Domain-Id: default Aug 27 02:44:00.401552 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-Project-Domain-Name: Default Aug 27 02:44:00.401552 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-Project-Id: 79e671941c234f36b542080d256de2c6 Aug 27 02:44:00.401552 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-Project-Name: tempest-TestVolumeBootPattern-1006469397 Aug 27 02:44:00.401552 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-Role: reader,member Aug 27 02:44:00.401552 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-Roles: reader,member Aug 27 02:44:00.403389 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-Service-Catalog: [{"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.52:8080", "publicURL": "http://10.222.0.52:8080/v1/AUTH_79e671941c234f36b542080d256de2c6"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52:9696/networking"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/compute/v2/79e671941c234f36b542080d256de2c6"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/image"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/volume/v3/79e671941c234f36b542080d256de2c6"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/identity"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/volume/v3/79e671941c234f36b542080d256de2c6"}]}] Aug 27 02:44:00.403389 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-Tenant: tempest-TestVolumeBootPattern-1006469397 Aug 27 02:44:00.403389 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-Tenant-Id: 79e671941c234f36b542080d256de2c6 Aug 27 02:44:00.403389 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-Tenant-Name: tempest-TestVolumeBootPattern-1006469397 Aug 27 02:44:00.403389 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-User: tempest-TestVolumeBootPattern-1006469397-project Aug 27 02:44:00.403389 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-User-Domain-Id: default Aug 27 02:44:00.403389 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-User-Domain-Name: Default Aug 27 02:44:00.403389 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-User-Id: 3fe5a1a037fa460eb3121e4d307542da Aug 27 02:44:00.403389 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-User-Name: tempest-TestVolumeBootPattern-1006469397-project Aug 27 02:44:00.403389 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: method: > Aug 27 02:44:00.403389 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: exception: Instance 2afc317e-fd8d-4e86-89d9-aba4b76d83cb could not be found. Aug 27 02:44:00.403389 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: action_args: {'id': '2afc317e-fd8d-4e86-89d9-aba4b76d83cb'} Aug 27 02:44:00.404478 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Aug 27 02:44:00.404478 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 27 02:44:00.404478 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Aug 27 02:44:00.404478 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Aug 27 02:44:00.404478 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Aug 27 02:44:00.404478 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Aug 27 02:44:00.404478 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Aug 27 02:44:00.404478 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Aug 27 02:44:00.404478 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Aug 27 02:44:00.404478 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Aug 27 02:44:00.404478 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 2afc317e-fd8d-4e86-89d9-aba4b76d83cb could not be found. Aug 27 02:44:00.404478 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi Aug 27 02:44:00.404478 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 27 02:44:00.404478 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi Aug 27 02:44:00.404478 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 27 02:44:00.404478 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 27 02:44:00.404478 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 27 02:44:00.404478 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Aug 27 02:44:00.404478 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Aug 27 02:44:00.404478 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Aug 27 02:44:00.406245 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Aug 27 02:44:00.406245 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Aug 27 02:44:00.406245 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi instance = self._get_server( Aug 27 02:44:00.406245 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Aug 27 02:44:00.406245 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Aug 27 02:44:00.406245 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Aug 27 02:44:00.406245 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Aug 27 02:44:00.406245 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 2afc317e-fd8d-4e86-89d9-aba4b76d83cb could not be found. Aug 27 02:44:00.406245 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi Aug 27 02:44:00.406245 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 27 02:44:00.406245 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi Aug 27 02:44:00.406245 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 27 02:44:00.406245 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 27 02:44:00.406245 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 27 02:44:00.406245 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Aug 27 02:44:00.406245 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Aug 27 02:44:00.406245 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 2afc317e-fd8d-4e86-89d9-aba4b76d83cb could not be found. Aug 27 02:44:00.406245 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi Aug 27 02:44:00.409087 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-2379f3a3-fbde-40a4-8547-974706eecd7a tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Returning 404 to user: Instance 2afc317e-fd8d-4e86-89d9-aba4b76d83cb could not be found. {{(pid=120864) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Aug 27 02:44:00.409087 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-2379f3a3-fbde-40a4-8547-974706eecd7a tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] 10.222.0.52 "GET /compute/v2.1/servers/2afc317e-fd8d-4e86-89d9-aba4b76d83cb" status: 404 len: 111 microversion: 2.1 time: 0.042263 Aug 27 02:44:00.409087 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 631/1253] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:44:00 2022] GET /compute/v2.1/servers/2afc317e-fd8d-4e86-89d9-aba4b76d83cb => generated 111 bytes in 45 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Aug 27 02:44:00.879085 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-c7e15e39-c08a-4fb0-a955-e3c0754e252c tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:44:00.879631 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-c7e15e39-c08a-4fb0-a955-e3c0754e252c tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] [instance: 30aa22c4-59b4-4482-b92a-3f386466f7ee] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:44:00.888463 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-c7e15e39-c08a-4fb0-a955-e3c0754e252c tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:44:00.889190 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-c7e15e39-c08a-4fb0-a955-e3c0754e252c tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:44:00.889753 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-c7e15e39-c08a-4fb0-a955-e3c0754e252c tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:44:00.890878 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-c7e15e39-c08a-4fb0-a955-e3c0754e252c tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:44:00.891598 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-c7e15e39-c08a-4fb0-a955-e3c0754e252c tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:44:00.892161 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-c7e15e39-c08a-4fb0-a955-e3c0754e252c tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:44:00.906000 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.wsgi [None req-c7e15e39-c08a-4fb0-a955-e3c0754e252c tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] HTTP exception thrown: Instance 30aa22c4-59b4-4482-b92a-3f386466f7ee could not be found. Aug 27 02:44:00.908145 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-c7e15e39-c08a-4fb0-a955-e3c0754e252c tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Request method failure captured: Aug 27 02:44:00.908145 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: request: DELETE /compute/v2.1/servers/30aa22c4-59b4-4482-b92a-3f386466f7ee HTTP/1.1 Aug 27 02:44:00.908145 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: Accept: application/json Aug 27 02:44:00.908145 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: Accept-Encoding: identity Aug 27 02:44:00.908145 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: Connection: close Aug 27 02:44:00.908145 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: Content-Length: 0 Aug 27 02:44:00.908145 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: Content-Type: application/json Aug 27 02:44:00.908145 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: Host: 10.222.0.52 Aug 27 02:44:00.908145 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: Openstack-System-Scope: None Aug 27 02:44:00.908145 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: User-Agent: python-urllib3/1.26.11 Aug 27 02:44:00.908145 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-Auth-Token: gAAAAABjCYLy85Uu3BiyLepTvyHrkionXI7siD38Rns3gISZZwndflckLvltMlTBHZJ6gk3GKZLgp1sOCr8IN2oWQwuE2aDeO0w7-eDMbQMQ90apHhdn2f8pHdjAco2Aqv2Vx1ThWF0nFT8Ht9XOqMF1ANor3utAF0ObK7gJJB5bN6WCMw2HLSw Aug 27 02:44:00.908145 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-Domain-Id: None Aug 27 02:44:00.908145 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-Domain-Name: None Aug 27 02:44:00.908145 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-Identity-Status: Confirmed Aug 27 02:44:00.908145 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-Is-Admin-Project: True Aug 27 02:44:00.908145 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-Project-Domain-Id: default Aug 27 02:44:00.908145 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-Project-Domain-Name: Default Aug 27 02:44:00.908145 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-Project-Id: 79e671941c234f36b542080d256de2c6 Aug 27 02:44:00.908145 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-Project-Name: tempest-TestVolumeBootPattern-1006469397 Aug 27 02:44:00.908145 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-Role: reader,member Aug 27 02:44:00.908145 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-Roles: reader,member Aug 27 02:44:00.909808 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-Service-Catalog: [{"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.52:8080", "publicURL": "http://10.222.0.52:8080/v1/AUTH_79e671941c234f36b542080d256de2c6"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52:9696/networking"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/compute/v2/79e671941c234f36b542080d256de2c6"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/image"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/volume/v3/79e671941c234f36b542080d256de2c6"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/identity"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/volume/v3/79e671941c234f36b542080d256de2c6"}]}] Aug 27 02:44:00.909808 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-Tenant: tempest-TestVolumeBootPattern-1006469397 Aug 27 02:44:00.909808 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-Tenant-Id: 79e671941c234f36b542080d256de2c6 Aug 27 02:44:00.909808 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-Tenant-Name: tempest-TestVolumeBootPattern-1006469397 Aug 27 02:44:00.909808 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-User: tempest-TestVolumeBootPattern-1006469397-project Aug 27 02:44:00.909808 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-User-Domain-Id: default Aug 27 02:44:00.909808 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-User-Domain-Name: Default Aug 27 02:44:00.909808 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-User-Id: 3fe5a1a037fa460eb3121e4d307542da Aug 27 02:44:00.909808 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-User-Name: tempest-TestVolumeBootPattern-1006469397-project Aug 27 02:44:00.909808 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: method: > Aug 27 02:44:00.909808 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: exception: Instance 30aa22c4-59b4-4482-b92a-3f386466f7ee could not be found. Aug 27 02:44:00.909808 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: action_args: {'id': '30aa22c4-59b4-4482-b92a-3f386466f7ee'} Aug 27 02:44:00.910619 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Aug 27 02:44:00.910619 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 27 02:44:00.910619 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Aug 27 02:44:00.910619 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Aug 27 02:44:00.910619 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Aug 27 02:44:00.910619 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Aug 27 02:44:00.910619 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Aug 27 02:44:00.910619 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Aug 27 02:44:00.910619 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Aug 27 02:44:00.910619 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Aug 27 02:44:00.910619 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 30aa22c4-59b4-4482-b92a-3f386466f7ee could not be found. Aug 27 02:44:00.910619 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi Aug 27 02:44:00.910619 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 27 02:44:00.910619 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi Aug 27 02:44:00.910619 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 27 02:44:00.910619 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 27 02:44:00.910619 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 27 02:44:00.910619 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Aug 27 02:44:00.910619 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Aug 27 02:44:00.910619 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Aug 27 02:44:00.911952 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Aug 27 02:44:00.911952 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Aug 27 02:44:00.911952 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Aug 27 02:44:00.911952 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Aug 27 02:44:00.911952 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Aug 27 02:44:00.911952 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Aug 27 02:44:00.911952 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Aug 27 02:44:00.911952 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Aug 27 02:44:00.911952 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Aug 27 02:44:00.911952 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 30aa22c4-59b4-4482-b92a-3f386466f7ee could not be found. Aug 27 02:44:00.911952 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi Aug 27 02:44:00.911952 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 27 02:44:00.911952 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi Aug 27 02:44:00.911952 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 27 02:44:00.911952 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 27 02:44:00.911952 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 27 02:44:00.911952 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Aug 27 02:44:00.911952 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Aug 27 02:44:00.911952 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 30aa22c4-59b4-4482-b92a-3f386466f7ee could not be found. Aug 27 02:44:00.911952 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi Aug 27 02:44:00.913385 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-c7e15e39-c08a-4fb0-a955-e3c0754e252c tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Returning 404 to user: Instance 30aa22c4-59b4-4482-b92a-3f386466f7ee could not be found. {{(pid=120863) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Aug 27 02:44:00.913385 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-c7e15e39-c08a-4fb0-a955-e3c0754e252c tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] 10.222.0.52 "DELETE /compute/v2.1/servers/30aa22c4-59b4-4482-b92a-3f386466f7ee" status: 404 len: 111 microversion: 2.1 time: 0.036975 Aug 27 02:44:00.913385 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 623/1254] 10.222.0.52 () {58 vars in 1239 bytes} [Sat Aug 27 02:44:00 2022] DELETE /compute/v2.1/servers/30aa22c4-59b4-4482-b92a-3f386466f7ee => generated 111 bytes in 38 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Aug 27 02:44:00.918886 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-fd97cbdb-c847-41f9-bff7-6332957f61a0 tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Calling method '>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:44:00.919550 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-fd97cbdb-c847-41f9-bff7-6332957f61a0 tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] [instance: 30aa22c4-59b4-4482-b92a-3f386466f7ee] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:44:00.926068 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-fd97cbdb-c847-41f9-bff7-6332957f61a0 tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:44:00.926551 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-fd97cbdb-c847-41f9-bff7-6332957f61a0 tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:44:00.926982 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-fd97cbdb-c847-41f9-bff7-6332957f61a0 tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:44:00.927707 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-fd97cbdb-c847-41f9-bff7-6332957f61a0 tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:44:00.928158 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-fd97cbdb-c847-41f9-bff7-6332957f61a0 tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:44:00.928578 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-fd97cbdb-c847-41f9-bff7-6332957f61a0 tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:44:00.944423 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.wsgi [None req-fd97cbdb-c847-41f9-bff7-6332957f61a0 tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] HTTP exception thrown: Instance 30aa22c4-59b4-4482-b92a-3f386466f7ee could not be found. Aug 27 02:44:00.946453 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-fd97cbdb-c847-41f9-bff7-6332957f61a0 tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Request method failure captured: Aug 27 02:44:00.946453 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: request: GET /compute/v2.1/servers/30aa22c4-59b4-4482-b92a-3f386466f7ee HTTP/1.1 Aug 27 02:44:00.946453 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: Accept: application/json Aug 27 02:44:00.946453 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: Accept-Encoding: identity Aug 27 02:44:00.946453 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: Connection: close Aug 27 02:44:00.946453 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: Content-Length: 0 Aug 27 02:44:00.946453 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: Content-Type: application/json Aug 27 02:44:00.946453 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: Host: 10.222.0.52 Aug 27 02:44:00.946453 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: Openstack-System-Scope: None Aug 27 02:44:00.946453 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: User-Agent: python-urllib3/1.26.11 Aug 27 02:44:00.946453 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-Auth-Token: gAAAAABjCYLy85Uu3BiyLepTvyHrkionXI7siD38Rns3gISZZwndflckLvltMlTBHZJ6gk3GKZLgp1sOCr8IN2oWQwuE2aDeO0w7-eDMbQMQ90apHhdn2f8pHdjAco2Aqv2Vx1ThWF0nFT8Ht9XOqMF1ANor3utAF0ObK7gJJB5bN6WCMw2HLSw Aug 27 02:44:00.946453 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-Domain-Id: None Aug 27 02:44:00.946453 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-Domain-Name: None Aug 27 02:44:00.946453 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-Identity-Status: Confirmed Aug 27 02:44:00.946453 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-Is-Admin-Project: True Aug 27 02:44:00.946453 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-Project-Domain-Id: default Aug 27 02:44:00.946453 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-Project-Domain-Name: Default Aug 27 02:44:00.946453 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-Project-Id: 79e671941c234f36b542080d256de2c6 Aug 27 02:44:00.946453 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-Project-Name: tempest-TestVolumeBootPattern-1006469397 Aug 27 02:44:00.946453 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-Role: reader,member Aug 27 02:44:00.946453 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-Roles: reader,member Aug 27 02:44:00.948443 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-Service-Catalog: [{"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.52:8080", "publicURL": "http://10.222.0.52:8080/v1/AUTH_79e671941c234f36b542080d256de2c6"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52:9696/networking"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/compute/v2/79e671941c234f36b542080d256de2c6"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/image"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/volume/v3/79e671941c234f36b542080d256de2c6"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/identity"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/volume/v3/79e671941c234f36b542080d256de2c6"}]}] Aug 27 02:44:00.948443 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-Tenant: tempest-TestVolumeBootPattern-1006469397 Aug 27 02:44:00.948443 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-Tenant-Id: 79e671941c234f36b542080d256de2c6 Aug 27 02:44:00.948443 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-Tenant-Name: tempest-TestVolumeBootPattern-1006469397 Aug 27 02:44:00.948443 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-User: tempest-TestVolumeBootPattern-1006469397-project Aug 27 02:44:00.948443 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-User-Domain-Id: default Aug 27 02:44:00.948443 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-User-Domain-Name: Default Aug 27 02:44:00.948443 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-User-Id: 3fe5a1a037fa460eb3121e4d307542da Aug 27 02:44:00.948443 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: X-User-Name: tempest-TestVolumeBootPattern-1006469397-project Aug 27 02:44:00.948443 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: method: > Aug 27 02:44:00.948443 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: exception: Instance 30aa22c4-59b4-4482-b92a-3f386466f7ee could not be found. Aug 27 02:44:00.948443 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: action_args: {'id': '30aa22c4-59b4-4482-b92a-3f386466f7ee'} Aug 27 02:44:00.949553 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Aug 27 02:44:00.949553 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 27 02:44:00.949553 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Aug 27 02:44:00.949553 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Aug 27 02:44:00.949553 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Aug 27 02:44:00.949553 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Aug 27 02:44:00.949553 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Aug 27 02:44:00.949553 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Aug 27 02:44:00.949553 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Aug 27 02:44:00.949553 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Aug 27 02:44:00.949553 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 30aa22c4-59b4-4482-b92a-3f386466f7ee could not be found. Aug 27 02:44:00.949553 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi Aug 27 02:44:00.949553 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 27 02:44:00.949553 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi Aug 27 02:44:00.949553 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 27 02:44:00.949553 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 27 02:44:00.949553 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 27 02:44:00.949553 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Aug 27 02:44:00.949553 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Aug 27 02:44:00.949553 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Aug 27 02:44:00.951295 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Aug 27 02:44:00.951295 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Aug 27 02:44:00.951295 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi instance = self._get_server( Aug 27 02:44:00.951295 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Aug 27 02:44:00.951295 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Aug 27 02:44:00.951295 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Aug 27 02:44:00.951295 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Aug 27 02:44:00.951295 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 30aa22c4-59b4-4482-b92a-3f386466f7ee could not be found. Aug 27 02:44:00.951295 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi Aug 27 02:44:00.951295 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 27 02:44:00.951295 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi Aug 27 02:44:00.951295 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 27 02:44:00.951295 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 27 02:44:00.951295 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 27 02:44:00.951295 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Aug 27 02:44:00.951295 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Aug 27 02:44:00.951295 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 30aa22c4-59b4-4482-b92a-3f386466f7ee could not be found. Aug 27 02:44:00.951295 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: ERROR nova.api.openstack.wsgi Aug 27 02:44:00.952835 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-fd97cbdb-c847-41f9-bff7-6332957f61a0 tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Returning 404 to user: Instance 30aa22c4-59b4-4482-b92a-3f386466f7ee could not be found. {{(pid=120864) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Aug 27 02:44:00.952835 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-fd97cbdb-c847-41f9-bff7-6332957f61a0 tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] 10.222.0.52 "GET /compute/v2.1/servers/30aa22c4-59b4-4482-b92a-3f386466f7ee" status: 404 len: 111 microversion: 2.1 time: 0.033168 Aug 27 02:44:00.952835 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 632/1255] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:44:00 2022] GET /compute/v2.1/servers/30aa22c4-59b4-4482-b92a-3f386466f7ee => generated 111 bytes in 35 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Aug 27 02:44:02.283302 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-316ce544-9305-40f4-9b6b-f07f211e8638 tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] Calling method '.version_select at 0x7f6b1b1e1670>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:44:02.310360 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-316ce544-9305-40f4-9b6b-f07f211e8638 tempest-TestVolumeBootPattern-1006469397 tempest-TestVolumeBootPattern-1006469397-project] 10.222.0.52 "DELETE /compute/v2.1/os-keypairs/tempest-TestVolumeBootPattern-1366561407" status: 202 len: 0 microversion: 2.1 time: 0.031714 Aug 27 02:44:02.311270 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 624/1256] 10.222.0.52 () {58 vars in 1263 bytes} [Sat Aug 27 02:44:02 2022] DELETE /compute/v2.1/os-keypairs/tempest-TestVolumeBootPattern-1366561407 => generated 0 bytes in 34 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Aug 27 02:44:44.367711 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-e393926f-c5f5-4ead-af8d-1034a92f1946 tempest-TestMinimumBasicScenario-1447275400 tempest-TestMinimumBasicScenario-1447275400-project] Action: 'action', calling method: >, body: {"reboot": {"type": "SOFT"}} {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 27 02:44:44.369994 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-e393926f-c5f5-4ead-af8d-1034a92f1946 tempest-TestMinimumBasicScenario-1447275400 tempest-TestMinimumBasicScenario-1447275400-project] [instance: 401ec9c3-4ff3-468a-b334-606c558b8d7a] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:44:44.372377 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG dbcounter [-] [120864] Writing DB stats nova_api:SELECT=18 {{(pid=120864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 27 02:44:44.373000 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG dbcounter [-] [120864] Writing DB stats nova_cell1:SELECT=26 {{(pid=120864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 27 02:44:44.379898 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-e393926f-c5f5-4ead-af8d-1034a92f1946 tempest-TestMinimumBasicScenario-1447275400 tempest-TestMinimumBasicScenario-1447275400-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:44:44.380384 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-e393926f-c5f5-4ead-af8d-1034a92f1946 tempest-TestMinimumBasicScenario-1447275400 tempest-TestMinimumBasicScenario-1447275400-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:44:44.380821 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-e393926f-c5f5-4ead-af8d-1034a92f1946 tempest-TestMinimumBasicScenario-1447275400 tempest-TestMinimumBasicScenario-1447275400-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:44:44.381539 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-e393926f-c5f5-4ead-af8d-1034a92f1946 tempest-TestMinimumBasicScenario-1447275400 tempest-TestMinimumBasicScenario-1447275400-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:44:44.381988 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-e393926f-c5f5-4ead-af8d-1034a92f1946 tempest-TestMinimumBasicScenario-1447275400 tempest-TestMinimumBasicScenario-1447275400-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:44:44.382405 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-e393926f-c5f5-4ead-af8d-1034a92f1946 tempest-TestMinimumBasicScenario-1447275400 tempest-TestMinimumBasicScenario-1447275400-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:44:44.464462 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-e393926f-c5f5-4ead-af8d-1034a92f1946 tempest-TestMinimumBasicScenario-1447275400 tempest-TestMinimumBasicScenario-1447275400-project] 10.222.0.52 "POST /compute/v2.1/servers/401ec9c3-4ff3-468a-b334-606c558b8d7a/action" status: 202 len: 0 microversion: 2.1 time: 0.100544 Aug 27 02:44:44.465309 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 633/1257] 10.222.0.52 () {60 vars in 1278 bytes} [Sat Aug 27 02:44:44 2022] POST /compute/v2.1/servers/401ec9c3-4ff3-468a-b334-606c558b8d7a/action => generated 0 bytes in 102 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Aug 27 02:44:44.473471 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-9bc4fcd6-6867-42a2-84cb-319ab61b16ef tempest-TestMinimumBasicScenario-1447275400 tempest-TestMinimumBasicScenario-1447275400-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:44:44.474160 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-9bc4fcd6-6867-42a2-84cb-319ab61b16ef tempest-TestMinimumBasicScenario-1447275400 tempest-TestMinimumBasicScenario-1447275400-project] [instance: 401ec9c3-4ff3-468a-b334-606c558b8d7a] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:44:44.476469 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG dbcounter [-] [120863] Writing DB stats nova_cell1:SELECT=54,nova_cell1:UPDATE=2,nova_cell1:INSERT=2 {{(pid=120863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 27 02:44:44.477279 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG dbcounter [-] [120863] Writing DB stats nova_api:SELECT=27,nova_api:UPDATE=2,nova_api:DELETE=1 {{(pid=120863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 27 02:44:44.485111 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-9bc4fcd6-6867-42a2-84cb-319ab61b16ef tempest-TestMinimumBasicScenario-1447275400 tempest-TestMinimumBasicScenario-1447275400-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:44:44.485715 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-9bc4fcd6-6867-42a2-84cb-319ab61b16ef tempest-TestMinimumBasicScenario-1447275400 tempest-TestMinimumBasicScenario-1447275400-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:44:44.486242 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-9bc4fcd6-6867-42a2-84cb-319ab61b16ef tempest-TestMinimumBasicScenario-1447275400 tempest-TestMinimumBasicScenario-1447275400-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:44:44.486969 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-9bc4fcd6-6867-42a2-84cb-319ab61b16ef tempest-TestMinimumBasicScenario-1447275400 tempest-TestMinimumBasicScenario-1447275400-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:44:44.487487 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-9bc4fcd6-6867-42a2-84cb-319ab61b16ef tempest-TestMinimumBasicScenario-1447275400 tempest-TestMinimumBasicScenario-1447275400-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:44:44.488102 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-9bc4fcd6-6867-42a2-84cb-319ab61b16ef tempest-TestMinimumBasicScenario-1447275400 tempest-TestMinimumBasicScenario-1447275400-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:44:44.744711 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-9bc4fcd6-6867-42a2-84cb-319ab61b16ef tempest-TestMinimumBasicScenario-1447275400 tempest-TestMinimumBasicScenario-1447275400-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ba73716667184588b522bc7856404be8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7dd5f20c4a714ee3b555d4aef8b87730', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:44:44.754383 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-9bc4fcd6-6867-42a2-84cb-319ab61b16ef tempest-TestMinimumBasicScenario-1447275400 tempest-TestMinimumBasicScenario-1447275400-project] 10.222.0.52 "GET /compute/v2.1/servers/401ec9c3-4ff3-468a-b334-606c558b8d7a" status: 200 len: 1751 microversion: 2.1 time: 0.284866 Aug 27 02:44:44.754965 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 625/1258] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:44:44 2022] GET /compute/v2.1/servers/401ec9c3-4ff3-468a-b334-606c558b8d7a => generated 1751 bytes in 286 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:44:45.779684 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-8ee13edf-fce3-4f36-abf3-94e2fcf44035 tempest-TestMinimumBasicScenario-1447275400 tempest-TestMinimumBasicScenario-1447275400-project] Calling method '>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:44:45.780954 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-8ee13edf-fce3-4f36-abf3-94e2fcf44035 tempest-TestMinimumBasicScenario-1447275400 tempest-TestMinimumBasicScenario-1447275400-project] [instance: 401ec9c3-4ff3-468a-b334-606c558b8d7a] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:44:45.789085 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-8ee13edf-fce3-4f36-abf3-94e2fcf44035 tempest-TestMinimumBasicScenario-1447275400 tempest-TestMinimumBasicScenario-1447275400-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:44:45.789593 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-8ee13edf-fce3-4f36-abf3-94e2fcf44035 tempest-TestMinimumBasicScenario-1447275400 tempest-TestMinimumBasicScenario-1447275400-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:44:45.790019 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-8ee13edf-fce3-4f36-abf3-94e2fcf44035 tempest-TestMinimumBasicScenario-1447275400 tempest-TestMinimumBasicScenario-1447275400-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:44:45.791003 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-8ee13edf-fce3-4f36-abf3-94e2fcf44035 tempest-TestMinimumBasicScenario-1447275400 tempest-TestMinimumBasicScenario-1447275400-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:44:45.791603 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-8ee13edf-fce3-4f36-abf3-94e2fcf44035 tempest-TestMinimumBasicScenario-1447275400 tempest-TestMinimumBasicScenario-1447275400-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:44:45.792208 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-8ee13edf-fce3-4f36-abf3-94e2fcf44035 tempest-TestMinimumBasicScenario-1447275400 tempest-TestMinimumBasicScenario-1447275400-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:44:45.972859 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-8ee13edf-fce3-4f36-abf3-94e2fcf44035 tempest-TestMinimumBasicScenario-1447275400 tempest-TestMinimumBasicScenario-1447275400-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ba73716667184588b522bc7856404be8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7dd5f20c4a714ee3b555d4aef8b87730', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:44:45.981511 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-8ee13edf-fce3-4f36-abf3-94e2fcf44035 tempest-TestMinimumBasicScenario-1447275400 tempest-TestMinimumBasicScenario-1447275400-project] 10.222.0.52 "GET /compute/v2.1/servers/401ec9c3-4ff3-468a-b334-606c558b8d7a" status: 200 len: 1756 microversion: 2.1 time: 0.208627 Aug 27 02:44:45.982071 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 634/1259] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:44:45 2022] GET /compute/v2.1/servers/401ec9c3-4ff3-468a-b334-606c558b8d7a => generated 1756 bytes in 210 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:44:47.005932 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-045d87fd-ee48-4dc1-bf6d-bcac1a347fa5 tempest-TestMinimumBasicScenario-1447275400 tempest-TestMinimumBasicScenario-1447275400-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:44:47.006526 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-045d87fd-ee48-4dc1-bf6d-bcac1a347fa5 tempest-TestMinimumBasicScenario-1447275400 tempest-TestMinimumBasicScenario-1447275400-project] [instance: 401ec9c3-4ff3-468a-b334-606c558b8d7a] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:44:47.015894 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-045d87fd-ee48-4dc1-bf6d-bcac1a347fa5 tempest-TestMinimumBasicScenario-1447275400 tempest-TestMinimumBasicScenario-1447275400-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:44:47.016380 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-045d87fd-ee48-4dc1-bf6d-bcac1a347fa5 tempest-TestMinimumBasicScenario-1447275400 tempest-TestMinimumBasicScenario-1447275400-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:44:47.016811 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-045d87fd-ee48-4dc1-bf6d-bcac1a347fa5 tempest-TestMinimumBasicScenario-1447275400 tempest-TestMinimumBasicScenario-1447275400-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:44:47.017566 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-045d87fd-ee48-4dc1-bf6d-bcac1a347fa5 tempest-TestMinimumBasicScenario-1447275400 tempest-TestMinimumBasicScenario-1447275400-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:44:47.017995 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-045d87fd-ee48-4dc1-bf6d-bcac1a347fa5 tempest-TestMinimumBasicScenario-1447275400 tempest-TestMinimumBasicScenario-1447275400-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:44:47.018414 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-045d87fd-ee48-4dc1-bf6d-bcac1a347fa5 tempest-TestMinimumBasicScenario-1447275400 tempest-TestMinimumBasicScenario-1447275400-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:44:47.181823 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-045d87fd-ee48-4dc1-bf6d-bcac1a347fa5 tempest-TestMinimumBasicScenario-1447275400 tempest-TestMinimumBasicScenario-1447275400-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ba73716667184588b522bc7856404be8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7dd5f20c4a714ee3b555d4aef8b87730', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:44:47.190309 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-045d87fd-ee48-4dc1-bf6d-bcac1a347fa5 tempest-TestMinimumBasicScenario-1447275400 tempest-TestMinimumBasicScenario-1447275400-project] 10.222.0.52 "GET /compute/v2.1/servers/401ec9c3-4ff3-468a-b334-606c558b8d7a" status: 200 len: 1756 microversion: 2.1 time: 0.189958 Aug 27 02:44:47.190858 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 626/1260] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:44:46 2022] GET /compute/v2.1/servers/401ec9c3-4ff3-468a-b334-606c558b8d7a => generated 1756 bytes in 191 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:44:48.219679 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-e86b3a3f-13b1-414a-af3c-1dd1f27a44cc tempest-TestMinimumBasicScenario-1447275400 tempest-TestMinimumBasicScenario-1447275400-project] Calling method '>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:44:48.220316 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-e86b3a3f-13b1-414a-af3c-1dd1f27a44cc tempest-TestMinimumBasicScenario-1447275400 tempest-TestMinimumBasicScenario-1447275400-project] [instance: 401ec9c3-4ff3-468a-b334-606c558b8d7a] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:44:48.223187 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Aug 27 02:44:48.223362 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: Sat Aug 27 02:44:48 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/401ec9c3-4ff3-468a-b334-606c558b8d7a (ip 10.222.0.52) !!! Aug 27 02:44:48.229744 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-e86b3a3f-13b1-414a-af3c-1dd1f27a44cc tempest-TestMinimumBasicScenario-1447275400 tempest-TestMinimumBasicScenario-1447275400-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:44:48.230227 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-e86b3a3f-13b1-414a-af3c-1dd1f27a44cc tempest-TestMinimumBasicScenario-1447275400 tempest-TestMinimumBasicScenario-1447275400-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:44:48.230655 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-e86b3a3f-13b1-414a-af3c-1dd1f27a44cc tempest-TestMinimumBasicScenario-1447275400 tempest-TestMinimumBasicScenario-1447275400-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:44:48.231345 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-e86b3a3f-13b1-414a-af3c-1dd1f27a44cc tempest-TestMinimumBasicScenario-1447275400 tempest-TestMinimumBasicScenario-1447275400-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:44:48.231789 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-e86b3a3f-13b1-414a-af3c-1dd1f27a44cc tempest-TestMinimumBasicScenario-1447275400 tempest-TestMinimumBasicScenario-1447275400-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:44:48.232210 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-e86b3a3f-13b1-414a-af3c-1dd1f27a44cc tempest-TestMinimumBasicScenario-1447275400 tempest-TestMinimumBasicScenario-1447275400-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:44:48.399787 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-e86b3a3f-13b1-414a-af3c-1dd1f27a44cc tempest-TestMinimumBasicScenario-1447275400 tempest-TestMinimumBasicScenario-1447275400-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ba73716667184588b522bc7856404be8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7dd5f20c4a714ee3b555d4aef8b87730', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:44:48.408976 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-e86b3a3f-13b1-414a-af3c-1dd1f27a44cc tempest-TestMinimumBasicScenario-1447275400 tempest-TestMinimumBasicScenario-1447275400-project] 10.222.0.52 "GET /compute/v2.1/servers/401ec9c3-4ff3-468a-b334-606c558b8d7a" status: 200 len: 1756 microversion: 2.1 time: 0.194372 Aug 27 02:44:48.409590 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 635/1261] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:44:48 2022] GET /compute/v2.1/servers/401ec9c3-4ff3-468a-b334-606c558b8d7a => generated 1756 bytes in 196 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:44:49.439831 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-769bce10-4c31-4a28-8391-87a4f6f9b1e4 tempest-TestMinimumBasicScenario-1447275400 tempest-TestMinimumBasicScenario-1447275400-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:44:49.440436 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-769bce10-4c31-4a28-8391-87a4f6f9b1e4 tempest-TestMinimumBasicScenario-1447275400 tempest-TestMinimumBasicScenario-1447275400-project] [instance: 401ec9c3-4ff3-468a-b334-606c558b8d7a] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:44:49.448428 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-769bce10-4c31-4a28-8391-87a4f6f9b1e4 tempest-TestMinimumBasicScenario-1447275400 tempest-TestMinimumBasicScenario-1447275400-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:44:49.448954 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-769bce10-4c31-4a28-8391-87a4f6f9b1e4 tempest-TestMinimumBasicScenario-1447275400 tempest-TestMinimumBasicScenario-1447275400-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:44:49.449411 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-769bce10-4c31-4a28-8391-87a4f6f9b1e4 tempest-TestMinimumBasicScenario-1447275400 tempest-TestMinimumBasicScenario-1447275400-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:44:49.450132 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-769bce10-4c31-4a28-8391-87a4f6f9b1e4 tempest-TestMinimumBasicScenario-1447275400 tempest-TestMinimumBasicScenario-1447275400-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:44:49.450609 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-769bce10-4c31-4a28-8391-87a4f6f9b1e4 tempest-TestMinimumBasicScenario-1447275400 tempest-TestMinimumBasicScenario-1447275400-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:44:49.451012 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-769bce10-4c31-4a28-8391-87a4f6f9b1e4 tempest-TestMinimumBasicScenario-1447275400 tempest-TestMinimumBasicScenario-1447275400-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:44:49.611051 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-769bce10-4c31-4a28-8391-87a4f6f9b1e4 tempest-TestMinimumBasicScenario-1447275400 tempest-TestMinimumBasicScenario-1447275400-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ba73716667184588b522bc7856404be8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7dd5f20c4a714ee3b555d4aef8b87730', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:44:49.621993 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-769bce10-4c31-4a28-8391-87a4f6f9b1e4 tempest-TestMinimumBasicScenario-1447275400 tempest-TestMinimumBasicScenario-1447275400-project] 10.222.0.52 "GET /compute/v2.1/servers/401ec9c3-4ff3-468a-b334-606c558b8d7a" status: 200 len: 1756 microversion: 2.1 time: 0.186405 Aug 27 02:44:49.622556 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 627/1262] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:44:49 2022] GET /compute/v2.1/servers/401ec9c3-4ff3-468a-b334-606c558b8d7a => generated 1756 bytes in 188 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 27 02:44:50.652088 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-211bdd63-5290-439b-b5fc-6b040e31e544 tempest-TestMinimumBasicScenario-1447275400 tempest-TestMinimumBasicScenario-1447275400-project] Calling method '>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:44:50.652784 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-211bdd63-5290-439b-b5fc-6b040e31e544 tempest-TestMinimumBasicScenario-1447275400 tempest-TestMinimumBasicScenario-1447275400-project] [instance: 401ec9c3-4ff3-468a-b334-606c558b8d7a] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:44:50.660916 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-211bdd63-5290-439b-b5fc-6b040e31e544 tempest-TestMinimumBasicScenario-1447275400 tempest-TestMinimumBasicScenario-1447275400-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:44:50.661438 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-211bdd63-5290-439b-b5fc-6b040e31e544 tempest-TestMinimumBasicScenario-1447275400 tempest-TestMinimumBasicScenario-1447275400-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:44:50.661877 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-211bdd63-5290-439b-b5fc-6b040e31e544 tempest-TestMinimumBasicScenario-1447275400 tempest-TestMinimumBasicScenario-1447275400-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:44:50.662567 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-211bdd63-5290-439b-b5fc-6b040e31e544 tempest-TestMinimumBasicScenario-1447275400 tempest-TestMinimumBasicScenario-1447275400-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:44:50.663008 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-211bdd63-5290-439b-b5fc-6b040e31e544 tempest-TestMinimumBasicScenario-1447275400 tempest-TestMinimumBasicScenario-1447275400-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:44:50.663514 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-211bdd63-5290-439b-b5fc-6b040e31e544 tempest-TestMinimumBasicScenario-1447275400 tempest-TestMinimumBasicScenario-1447275400-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:44:50.834588 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-211bdd63-5290-439b-b5fc-6b040e31e544 tempest-TestMinimumBasicScenario-1447275400 tempest-TestMinimumBasicScenario-1447275400-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ba73716667184588b522bc7856404be8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7dd5f20c4a714ee3b555d4aef8b87730', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:44:50.843956 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-211bdd63-5290-439b-b5fc-6b040e31e544 tempest-TestMinimumBasicScenario-1447275400 tempest-TestMinimumBasicScenario-1447275400-project] 10.222.0.52 "GET /compute/v2.1/servers/401ec9c3-4ff3-468a-b334-606c558b8d7a" status: 200 len: 1756 microversion: 2.1 time: 0.195722 Aug 27 02:44:50.844536 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 636/1263] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:44:50 2022] GET /compute/v2.1/servers/401ec9c3-4ff3-468a-b334-606c558b8d7a => generated 1756 bytes in 197 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:44:51.868142 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-acda13d4-9a85-4e96-9a94-66313265b2f1 tempest-TestMinimumBasicScenario-1447275400 tempest-TestMinimumBasicScenario-1447275400-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:44:51.868758 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-acda13d4-9a85-4e96-9a94-66313265b2f1 tempest-TestMinimumBasicScenario-1447275400 tempest-TestMinimumBasicScenario-1447275400-project] [instance: 401ec9c3-4ff3-468a-b334-606c558b8d7a] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:44:51.875455 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-acda13d4-9a85-4e96-9a94-66313265b2f1 tempest-TestMinimumBasicScenario-1447275400 tempest-TestMinimumBasicScenario-1447275400-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:44:51.875944 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-acda13d4-9a85-4e96-9a94-66313265b2f1 tempest-TestMinimumBasicScenario-1447275400 tempest-TestMinimumBasicScenario-1447275400-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:44:51.876382 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-acda13d4-9a85-4e96-9a94-66313265b2f1 tempest-TestMinimumBasicScenario-1447275400 tempest-TestMinimumBasicScenario-1447275400-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:44:51.877079 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-acda13d4-9a85-4e96-9a94-66313265b2f1 tempest-TestMinimumBasicScenario-1447275400 tempest-TestMinimumBasicScenario-1447275400-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:44:51.877560 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-acda13d4-9a85-4e96-9a94-66313265b2f1 tempest-TestMinimumBasicScenario-1447275400 tempest-TestMinimumBasicScenario-1447275400-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:44:51.878023 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-acda13d4-9a85-4e96-9a94-66313265b2f1 tempest-TestMinimumBasicScenario-1447275400 tempest-TestMinimumBasicScenario-1447275400-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:44:52.023617 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-acda13d4-9a85-4e96-9a94-66313265b2f1 tempest-TestMinimumBasicScenario-1447275400 tempest-TestMinimumBasicScenario-1447275400-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ba73716667184588b522bc7856404be8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7dd5f20c4a714ee3b555d4aef8b87730', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:44:52.032692 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-acda13d4-9a85-4e96-9a94-66313265b2f1 tempest-TestMinimumBasicScenario-1447275400 tempest-TestMinimumBasicScenario-1447275400-project] 10.222.0.52 "GET /compute/v2.1/servers/401ec9c3-4ff3-468a-b334-606c558b8d7a" status: 200 len: 1756 microversion: 2.1 time: 0.169055 Aug 27 02:44:52.033341 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 628/1264] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:44:51 2022] GET /compute/v2.1/servers/401ec9c3-4ff3-468a-b334-606c558b8d7a => generated 1756 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:44:53.055341 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-a15f41fc-0242-4df3-aa7e-a1feab0e1dcc tempest-TestMinimumBasicScenario-1447275400 tempest-TestMinimumBasicScenario-1447275400-project] Calling method '>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:44:53.056022 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-a15f41fc-0242-4df3-aa7e-a1feab0e1dcc tempest-TestMinimumBasicScenario-1447275400 tempest-TestMinimumBasicScenario-1447275400-project] [instance: 401ec9c3-4ff3-468a-b334-606c558b8d7a] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:44:53.063402 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-a15f41fc-0242-4df3-aa7e-a1feab0e1dcc tempest-TestMinimumBasicScenario-1447275400 tempest-TestMinimumBasicScenario-1447275400-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:44:53.063882 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-a15f41fc-0242-4df3-aa7e-a1feab0e1dcc tempest-TestMinimumBasicScenario-1447275400 tempest-TestMinimumBasicScenario-1447275400-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:44:53.064304 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-a15f41fc-0242-4df3-aa7e-a1feab0e1dcc tempest-TestMinimumBasicScenario-1447275400 tempest-TestMinimumBasicScenario-1447275400-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:44:53.064994 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-a15f41fc-0242-4df3-aa7e-a1feab0e1dcc tempest-TestMinimumBasicScenario-1447275400 tempest-TestMinimumBasicScenario-1447275400-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:44:53.065469 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-a15f41fc-0242-4df3-aa7e-a1feab0e1dcc tempest-TestMinimumBasicScenario-1447275400 tempest-TestMinimumBasicScenario-1447275400-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:44:53.065911 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-a15f41fc-0242-4df3-aa7e-a1feab0e1dcc tempest-TestMinimumBasicScenario-1447275400 tempest-TestMinimumBasicScenario-1447275400-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:44:53.210690 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-a15f41fc-0242-4df3-aa7e-a1feab0e1dcc tempest-TestMinimumBasicScenario-1447275400 tempest-TestMinimumBasicScenario-1447275400-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ba73716667184588b522bc7856404be8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7dd5f20c4a714ee3b555d4aef8b87730', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:44:53.219894 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-a15f41fc-0242-4df3-aa7e-a1feab0e1dcc tempest-TestMinimumBasicScenario-1447275400 tempest-TestMinimumBasicScenario-1447275400-project] 10.222.0.52 "GET /compute/v2.1/servers/401ec9c3-4ff3-468a-b334-606c558b8d7a" status: 200 len: 1756 microversion: 2.1 time: 0.168523 Aug 27 02:44:53.220467 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 637/1265] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:44:53 2022] GET /compute/v2.1/servers/401ec9c3-4ff3-468a-b334-606c558b8d7a => generated 1756 bytes in 170 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:44:54.242250 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-1cd458f3-60ab-48fd-a061-907d6f880514 tempest-TestMinimumBasicScenario-1447275400 tempest-TestMinimumBasicScenario-1447275400-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:44:54.243103 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-1cd458f3-60ab-48fd-a061-907d6f880514 tempest-TestMinimumBasicScenario-1447275400 tempest-TestMinimumBasicScenario-1447275400-project] [instance: 401ec9c3-4ff3-468a-b334-606c558b8d7a] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:44:54.249849 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-1cd458f3-60ab-48fd-a061-907d6f880514 tempest-TestMinimumBasicScenario-1447275400 tempest-TestMinimumBasicScenario-1447275400-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:44:54.250324 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-1cd458f3-60ab-48fd-a061-907d6f880514 tempest-TestMinimumBasicScenario-1447275400 tempest-TestMinimumBasicScenario-1447275400-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:44:54.250757 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-1cd458f3-60ab-48fd-a061-907d6f880514 tempest-TestMinimumBasicScenario-1447275400 tempest-TestMinimumBasicScenario-1447275400-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:44:54.251469 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-1cd458f3-60ab-48fd-a061-907d6f880514 tempest-TestMinimumBasicScenario-1447275400 tempest-TestMinimumBasicScenario-1447275400-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:44:54.251911 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-1cd458f3-60ab-48fd-a061-907d6f880514 tempest-TestMinimumBasicScenario-1447275400 tempest-TestMinimumBasicScenario-1447275400-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:44:54.252328 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-1cd458f3-60ab-48fd-a061-907d6f880514 tempest-TestMinimumBasicScenario-1447275400 tempest-TestMinimumBasicScenario-1447275400-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:44:54.408360 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-1cd458f3-60ab-48fd-a061-907d6f880514 tempest-TestMinimumBasicScenario-1447275400 tempest-TestMinimumBasicScenario-1447275400-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ba73716667184588b522bc7856404be8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7dd5f20c4a714ee3b555d4aef8b87730', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:44:54.416938 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-1cd458f3-60ab-48fd-a061-907d6f880514 tempest-TestMinimumBasicScenario-1447275400 tempest-TestMinimumBasicScenario-1447275400-project] 10.222.0.52 "GET /compute/v2.1/servers/401ec9c3-4ff3-468a-b334-606c558b8d7a" status: 200 len: 1756 microversion: 2.1 time: 0.178767 Aug 27 02:44:54.417545 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 629/1266] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:44:54 2022] GET /compute/v2.1/servers/401ec9c3-4ff3-468a-b334-606c558b8d7a => generated 1756 bytes in 180 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:44:55.439414 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-9dedca45-1177-43c6-8c12-b5729db4e69e tempest-TestMinimumBasicScenario-1447275400 tempest-TestMinimumBasicScenario-1447275400-project] Calling method '>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:44:55.439996 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-9dedca45-1177-43c6-8c12-b5729db4e69e tempest-TestMinimumBasicScenario-1447275400 tempest-TestMinimumBasicScenario-1447275400-project] [instance: 401ec9c3-4ff3-468a-b334-606c558b8d7a] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:44:55.446718 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-9dedca45-1177-43c6-8c12-b5729db4e69e tempest-TestMinimumBasicScenario-1447275400 tempest-TestMinimumBasicScenario-1447275400-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:44:55.447226 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-9dedca45-1177-43c6-8c12-b5729db4e69e tempest-TestMinimumBasicScenario-1447275400 tempest-TestMinimumBasicScenario-1447275400-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:44:55.447654 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-9dedca45-1177-43c6-8c12-b5729db4e69e tempest-TestMinimumBasicScenario-1447275400 tempest-TestMinimumBasicScenario-1447275400-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:44:55.448351 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-9dedca45-1177-43c6-8c12-b5729db4e69e tempest-TestMinimumBasicScenario-1447275400 tempest-TestMinimumBasicScenario-1447275400-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:44:55.448802 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-9dedca45-1177-43c6-8c12-b5729db4e69e tempest-TestMinimumBasicScenario-1447275400 tempest-TestMinimumBasicScenario-1447275400-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:44:55.449252 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-9dedca45-1177-43c6-8c12-b5729db4e69e tempest-TestMinimumBasicScenario-1447275400 tempest-TestMinimumBasicScenario-1447275400-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:44:55.614239 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-9dedca45-1177-43c6-8c12-b5729db4e69e tempest-TestMinimumBasicScenario-1447275400 tempest-TestMinimumBasicScenario-1447275400-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ba73716667184588b522bc7856404be8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7dd5f20c4a714ee3b555d4aef8b87730', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:44:55.622851 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-9dedca45-1177-43c6-8c12-b5729db4e69e tempest-TestMinimumBasicScenario-1447275400 tempest-TestMinimumBasicScenario-1447275400-project] 10.222.0.52 "GET /compute/v2.1/servers/401ec9c3-4ff3-468a-b334-606c558b8d7a" status: 200 len: 1756 microversion: 2.1 time: 0.187959 Aug 27 02:44:55.623668 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 638/1267] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:44:55 2022] GET /compute/v2.1/servers/401ec9c3-4ff3-468a-b334-606c558b8d7a => generated 1756 bytes in 189 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:44:56.648654 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-ce0ab0de-9b6d-4b09-8015-d5e440b79b74 tempest-TestMinimumBasicScenario-1447275400 tempest-TestMinimumBasicScenario-1447275400-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:44:56.649314 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-ce0ab0de-9b6d-4b09-8015-d5e440b79b74 tempest-TestMinimumBasicScenario-1447275400 tempest-TestMinimumBasicScenario-1447275400-project] [instance: 401ec9c3-4ff3-468a-b334-606c558b8d7a] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:44:56.657842 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-ce0ab0de-9b6d-4b09-8015-d5e440b79b74 tempest-TestMinimumBasicScenario-1447275400 tempest-TestMinimumBasicScenario-1447275400-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:44:56.658335 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-ce0ab0de-9b6d-4b09-8015-d5e440b79b74 tempest-TestMinimumBasicScenario-1447275400 tempest-TestMinimumBasicScenario-1447275400-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:44:56.658781 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-ce0ab0de-9b6d-4b09-8015-d5e440b79b74 tempest-TestMinimumBasicScenario-1447275400 tempest-TestMinimumBasicScenario-1447275400-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:44:56.659525 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-ce0ab0de-9b6d-4b09-8015-d5e440b79b74 tempest-TestMinimumBasicScenario-1447275400 tempest-TestMinimumBasicScenario-1447275400-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:44:56.660010 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-ce0ab0de-9b6d-4b09-8015-d5e440b79b74 tempest-TestMinimumBasicScenario-1447275400 tempest-TestMinimumBasicScenario-1447275400-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:44:56.660437 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-ce0ab0de-9b6d-4b09-8015-d5e440b79b74 tempest-TestMinimumBasicScenario-1447275400 tempest-TestMinimumBasicScenario-1447275400-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:44:56.811166 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-ce0ab0de-9b6d-4b09-8015-d5e440b79b74 tempest-TestMinimumBasicScenario-1447275400 tempest-TestMinimumBasicScenario-1447275400-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ba73716667184588b522bc7856404be8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7dd5f20c4a714ee3b555d4aef8b87730', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:44:56.820198 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-ce0ab0de-9b6d-4b09-8015-d5e440b79b74 tempest-TestMinimumBasicScenario-1447275400 tempest-TestMinimumBasicScenario-1447275400-project] 10.222.0.52 "GET /compute/v2.1/servers/401ec9c3-4ff3-468a-b334-606c558b8d7a" status: 200 len: 1759 microversion: 2.1 time: 0.177948 Aug 27 02:44:56.820774 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 630/1268] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:44:56 2022] GET /compute/v2.1/servers/401ec9c3-4ff3-468a-b334-606c558b8d7a => generated 1759 bytes in 180 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:46:00.160867 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG dbcounter [-] [120864] Writing DB stats nova_api:SELECT=12 {{(pid=120864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 27 02:46:00.161656 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG dbcounter [-] [120864] Writing DB stats nova_cell1:SELECT=40,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=120864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 27 02:46:00.165767 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-c4dc8716-7f19-4a3f-9258-532a13b69dfc tempest-TestMinimumBasicScenario-1447275400 tempest-TestMinimumBasicScenario-1447275400-project] Action: 'action', calling method: >, body: {"removeSecurityGroup": {"name": "tempest-secgroup-smoke-1539641839"}} {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 27 02:46:00.166318 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-c4dc8716-7f19-4a3f-9258-532a13b69dfc tempest-TestMinimumBasicScenario-1447275400 tempest-TestMinimumBasicScenario-1447275400-project] [instance: 401ec9c3-4ff3-468a-b334-606c558b8d7a] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:46:00.173771 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-c4dc8716-7f19-4a3f-9258-532a13b69dfc tempest-TestMinimumBasicScenario-1447275400 tempest-TestMinimumBasicScenario-1447275400-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:46:00.174253 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-c4dc8716-7f19-4a3f-9258-532a13b69dfc tempest-TestMinimumBasicScenario-1447275400 tempest-TestMinimumBasicScenario-1447275400-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:46:00.174684 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-c4dc8716-7f19-4a3f-9258-532a13b69dfc tempest-TestMinimumBasicScenario-1447275400 tempest-TestMinimumBasicScenario-1447275400-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:46:00.175377 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-c4dc8716-7f19-4a3f-9258-532a13b69dfc tempest-TestMinimumBasicScenario-1447275400 tempest-TestMinimumBasicScenario-1447275400-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:46:00.175826 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-c4dc8716-7f19-4a3f-9258-532a13b69dfc tempest-TestMinimumBasicScenario-1447275400 tempest-TestMinimumBasicScenario-1447275400-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:46:00.176245 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-c4dc8716-7f19-4a3f-9258-532a13b69dfc tempest-TestMinimumBasicScenario-1447275400 tempest-TestMinimumBasicScenario-1447275400-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:46:00.323735 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.network.security_group_api [None req-c4dc8716-7f19-4a3f-9258-532a13b69dfc tempest-TestMinimumBasicScenario-1447275400 tempest-TestMinimumBasicScenario-1447275400-project] Removing security group a4941718-8a89-4358-b890-d2d55433ecb9 from port 974dc3f6-d8bf-4ab5-a284-a4ecb61b8866 Aug 27 02:46:00.647970 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-c4dc8716-7f19-4a3f-9258-532a13b69dfc tempest-TestMinimumBasicScenario-1447275400 tempest-TestMinimumBasicScenario-1447275400-project] 10.222.0.52 "POST /compute/v2.1/servers/401ec9c3-4ff3-468a-b334-606c558b8d7a/action" status: 202 len: 0 microversion: 2.1 time: 0.488922 Aug 27 02:46:00.648615 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 639/1269] 10.222.0.52 () {60 vars in 1278 bytes} [Sat Aug 27 02:46:00 2022] POST /compute/v2.1/servers/401ec9c3-4ff3-468a-b334-606c558b8d7a/action => generated 0 bytes in 490 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Aug 27 02:46:00.655513 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG dbcounter [-] [120863] Writing DB stats nova_api:SELECT=12 {{(pid=120863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 27 02:46:00.663027 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG dbcounter [-] [120863] Writing DB stats nova_cell1:SELECT=36 {{(pid=120863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 27 02:46:00.665100 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Aug 27 02:46:00.665439 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: Sat Aug 27 02:46:00 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/os-server-external-events (ip 10.222.0.52) !!! Aug 27 02:46:00.673263 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [req-609842c7-ea61-48c0-a570-df2b460410e7 req-178e93ed-6322-4d81-8f4e-eb3fa5df7b9a service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "401ec9c3-4ff3-468a-b334-606c558b8d7a", "tag": "974dc3f6-d8bf-4ab5-a284-a4ecb61b8866"}]} {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 27 02:46:00.683281 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [req-609842c7-ea61-48c0-a570-df2b460410e7 req-178e93ed-6322-4d81-8f4e-eb3fa5df7b9a service nova] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:46:00.683937 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [req-609842c7-ea61-48c0-a570-df2b460410e7 req-178e93ed-6322-4d81-8f4e-eb3fa5df7b9a service nova] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:46:00.684377 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [req-609842c7-ea61-48c0-a570-df2b460410e7 req-178e93ed-6322-4d81-8f4e-eb3fa5df7b9a service nova] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:46:00.697303 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.compute.server_external_events [req-609842c7-ea61-48c0-a570-df2b460410e7 req-178e93ed-6322-4d81-8f4e-eb3fa5df7b9a service nova] Creating event network-changed:974dc3f6-d8bf-4ab5-a284-a4ecb61b8866 for instance 401ec9c3-4ff3-468a-b334-606c558b8d7a on hv-cinder-84630 Aug 27 02:46:00.703551 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [req-609842c7-ea61-48c0-a570-df2b460410e7 req-178e93ed-6322-4d81-8f4e-eb3fa5df7b9a service nova] 10.222.0.52 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.050768 Aug 27 02:46:00.705071 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 631/1270] 10.222.0.52 () {64 vars in 1317 bytes} [Sat Aug 27 02:46:00 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 54 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 27 02:46:01.982214 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-522746f2-d4ae-45c8-bfd8-a14651d322e5 tempest-TestMinimumBasicScenario-1447275400 tempest-TestMinimumBasicScenario-1447275400-project] Calling method '>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:46:01.982922 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-522746f2-d4ae-45c8-bfd8-a14651d322e5 tempest-TestMinimumBasicScenario-1447275400 tempest-TestMinimumBasicScenario-1447275400-project] [instance: 401ec9c3-4ff3-468a-b334-606c558b8d7a] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:46:01.990324 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-522746f2-d4ae-45c8-bfd8-a14651d322e5 tempest-TestMinimumBasicScenario-1447275400 tempest-TestMinimumBasicScenario-1447275400-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:46:01.991258 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-522746f2-d4ae-45c8-bfd8-a14651d322e5 tempest-TestMinimumBasicScenario-1447275400 tempest-TestMinimumBasicScenario-1447275400-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:46:01.991430 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [req-742074cf-3708-4d4e-83ca-1754cbbe7156 req-ebb81e6a-f0bd-4e4b-8c6d-6b2a7a0b04a2 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "401ec9c3-4ff3-468a-b334-606c558b8d7a", "tag": "974dc3f6-d8bf-4ab5-a284-a4ecb61b8866"}]} {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 27 02:46:01.992449 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-522746f2-d4ae-45c8-bfd8-a14651d322e5 tempest-TestMinimumBasicScenario-1447275400 tempest-TestMinimumBasicScenario-1447275400-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:46:01.993217 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-522746f2-d4ae-45c8-bfd8-a14651d322e5 tempest-TestMinimumBasicScenario-1447275400 tempest-TestMinimumBasicScenario-1447275400-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:46:01.993827 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-522746f2-d4ae-45c8-bfd8-a14651d322e5 tempest-TestMinimumBasicScenario-1447275400 tempest-TestMinimumBasicScenario-1447275400-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:46:01.994405 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-522746f2-d4ae-45c8-bfd8-a14651d322e5 tempest-TestMinimumBasicScenario-1447275400 tempest-TestMinimumBasicScenario-1447275400-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:46:02.005606 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [req-742074cf-3708-4d4e-83ca-1754cbbe7156 req-ebb81e6a-f0bd-4e4b-8c6d-6b2a7a0b04a2 service nova] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:46:02.006512 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [req-742074cf-3708-4d4e-83ca-1754cbbe7156 req-ebb81e6a-f0bd-4e4b-8c6d-6b2a7a0b04a2 service nova] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:46:02.006953 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [req-742074cf-3708-4d4e-83ca-1754cbbe7156 req-ebb81e6a-f0bd-4e4b-8c6d-6b2a7a0b04a2 service nova] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:46:02.019624 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.compute.server_external_events [req-742074cf-3708-4d4e-83ca-1754cbbe7156 req-ebb81e6a-f0bd-4e4b-8c6d-6b2a7a0b04a2 service nova] Creating event network-changed:974dc3f6-d8bf-4ab5-a284-a4ecb61b8866 for instance 401ec9c3-4ff3-468a-b334-606c558b8d7a on hv-cinder-84630 Aug 27 02:46:02.027290 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [req-742074cf-3708-4d4e-83ca-1754cbbe7156 req-ebb81e6a-f0bd-4e4b-8c6d-6b2a7a0b04a2 service nova] 10.222.0.52 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.044088 Aug 27 02:46:02.028311 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 632/1271] 10.222.0.52 () {64 vars in 1317 bytes} [Sat Aug 27 02:46:01 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 51 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Aug 27 02:46:02.094938 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.objects.instance [None req-522746f2-d4ae-45c8-bfd8-a14651d322e5 tempest-TestMinimumBasicScenario-1447275400 tempest-TestMinimumBasicScenario-1447275400-project] Lazy-loading 'services' on Instance uuid 401ec9c3-4ff3-468a-b334-606c558b8d7a {{(pid=120864) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Aug 27 02:46:02.178298 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-522746f2-d4ae-45c8-bfd8-a14651d322e5 tempest-TestMinimumBasicScenario-1447275400 tempest-TestMinimumBasicScenario-1447275400-project] 10.222.0.52 "DELETE /compute/v2.1/servers/401ec9c3-4ff3-468a-b334-606c558b8d7a/os-volume_attachments/f3c255b5-9631-420b-ad9d-b251e58e39ae" status: 202 len: 0 microversion: 2.1 time: 0.203446 Aug 27 02:46:02.178926 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 640/1272] 10.222.0.52 () {58 vars in 1416 bytes} [Sat Aug 27 02:46:01 2022] DELETE /compute/v2.1/servers/401ec9c3-4ff3-468a-b334-606c558b8d7a/os-volume_attachments/f3c255b5-9631-420b-ad9d-b251e58e39ae => generated 0 bytes in 206 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Aug 27 02:46:02.188941 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-aea91b27-bbbd-410f-bbdc-9b43df20d24a tempest-TestMinimumBasicScenario-1447275400 tempest-TestMinimumBasicScenario-1447275400-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:46:02.191564 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-aea91b27-bbbd-410f-bbdc-9b43df20d24a tempest-TestMinimumBasicScenario-1447275400 tempest-TestMinimumBasicScenario-1447275400-project] [instance: 401ec9c3-4ff3-468a-b334-606c558b8d7a] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:46:02.198592 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-aea91b27-bbbd-410f-bbdc-9b43df20d24a tempest-TestMinimumBasicScenario-1447275400 tempest-TestMinimumBasicScenario-1447275400-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:46:02.199424 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-aea91b27-bbbd-410f-bbdc-9b43df20d24a tempest-TestMinimumBasicScenario-1447275400 tempest-TestMinimumBasicScenario-1447275400-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:46:02.200180 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-aea91b27-bbbd-410f-bbdc-9b43df20d24a tempest-TestMinimumBasicScenario-1447275400 tempest-TestMinimumBasicScenario-1447275400-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:46:02.201097 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-aea91b27-bbbd-410f-bbdc-9b43df20d24a tempest-TestMinimumBasicScenario-1447275400 tempest-TestMinimumBasicScenario-1447275400-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:46:02.201767 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-aea91b27-bbbd-410f-bbdc-9b43df20d24a tempest-TestMinimumBasicScenario-1447275400 tempest-TestMinimumBasicScenario-1447275400-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:46:02.202402 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-aea91b27-bbbd-410f-bbdc-9b43df20d24a tempest-TestMinimumBasicScenario-1447275400 tempest-TestMinimumBasicScenario-1447275400-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:46:02.247546 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-aea91b27-bbbd-410f-bbdc-9b43df20d24a tempest-TestMinimumBasicScenario-1447275400 tempest-TestMinimumBasicScenario-1447275400-project] 10.222.0.52 "GET /compute/v2.1/servers/401ec9c3-4ff3-468a-b334-606c558b8d7a/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.062960 Aug 27 02:46:02.253824 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 633/1273] 10.222.0.52 () {58 vars in 1302 bytes} [Sat Aug 27 02:46:02 2022] GET /compute/v2.1/servers/401ec9c3-4ff3-468a-b334-606c558b8d7a/os-volume_attachments => generated 197 bytes in 66 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Aug 27 02:46:03.263143 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-87f2f09e-4681-4daa-8944-a9c3bca23260 tempest-TestMinimumBasicScenario-1447275400 tempest-TestMinimumBasicScenario-1447275400-project] Calling method '>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:46:03.265613 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-87f2f09e-4681-4daa-8944-a9c3bca23260 tempest-TestMinimumBasicScenario-1447275400 tempest-TestMinimumBasicScenario-1447275400-project] [instance: 401ec9c3-4ff3-468a-b334-606c558b8d7a] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:46:03.273289 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-87f2f09e-4681-4daa-8944-a9c3bca23260 tempest-TestMinimumBasicScenario-1447275400 tempest-TestMinimumBasicScenario-1447275400-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:46:03.273901 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-87f2f09e-4681-4daa-8944-a9c3bca23260 tempest-TestMinimumBasicScenario-1447275400 tempest-TestMinimumBasicScenario-1447275400-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:46:03.274404 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-87f2f09e-4681-4daa-8944-a9c3bca23260 tempest-TestMinimumBasicScenario-1447275400 tempest-TestMinimumBasicScenario-1447275400-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:46:03.275185 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-87f2f09e-4681-4daa-8944-a9c3bca23260 tempest-TestMinimumBasicScenario-1447275400 tempest-TestMinimumBasicScenario-1447275400-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:46:03.275711 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-87f2f09e-4681-4daa-8944-a9c3bca23260 tempest-TestMinimumBasicScenario-1447275400 tempest-TestMinimumBasicScenario-1447275400-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:46:03.276143 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-87f2f09e-4681-4daa-8944-a9c3bca23260 tempest-TestMinimumBasicScenario-1447275400 tempest-TestMinimumBasicScenario-1447275400-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:46:03.312874 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-87f2f09e-4681-4daa-8944-a9c3bca23260 tempest-TestMinimumBasicScenario-1447275400 tempest-TestMinimumBasicScenario-1447275400-project] 10.222.0.52 "GET /compute/v2.1/servers/401ec9c3-4ff3-468a-b334-606c558b8d7a/os-volume_attachments" status: 200 len: 25 microversion: 2.1 time: 0.054141 Aug 27 02:46:03.314169 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 641/1274] 10.222.0.52 () {58 vars in 1302 bytes} [Sat Aug 27 02:46:03 2022] GET /compute/v2.1/servers/401ec9c3-4ff3-468a-b334-606c558b8d7a/os-volume_attachments => generated 25 bytes in 57 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Aug 27 02:46:03.325715 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-fb5eb7c9-7ef3-4b74-a509-af9e70908fd3 tempest-TestMinimumBasicScenario-1447275400 tempest-TestMinimumBasicScenario-1447275400-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:46:03.326595 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-fb5eb7c9-7ef3-4b74-a509-af9e70908fd3 tempest-TestMinimumBasicScenario-1447275400 tempest-TestMinimumBasicScenario-1447275400-project] [instance: 401ec9c3-4ff3-468a-b334-606c558b8d7a] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:46:03.333503 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-fb5eb7c9-7ef3-4b74-a509-af9e70908fd3 tempest-TestMinimumBasicScenario-1447275400 tempest-TestMinimumBasicScenario-1447275400-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:46:03.333944 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-fb5eb7c9-7ef3-4b74-a509-af9e70908fd3 tempest-TestMinimumBasicScenario-1447275400 tempest-TestMinimumBasicScenario-1447275400-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:46:03.334374 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-fb5eb7c9-7ef3-4b74-a509-af9e70908fd3 tempest-TestMinimumBasicScenario-1447275400 tempest-TestMinimumBasicScenario-1447275400-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:46:03.335060 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-fb5eb7c9-7ef3-4b74-a509-af9e70908fd3 tempest-TestMinimumBasicScenario-1447275400 tempest-TestMinimumBasicScenario-1447275400-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:46:03.335506 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-fb5eb7c9-7ef3-4b74-a509-af9e70908fd3 tempest-TestMinimumBasicScenario-1447275400 tempest-TestMinimumBasicScenario-1447275400-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:46:03.335919 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-fb5eb7c9-7ef3-4b74-a509-af9e70908fd3 tempest-TestMinimumBasicScenario-1447275400 tempest-TestMinimumBasicScenario-1447275400-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:46:03.412657 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.wsgi [None req-fb5eb7c9-7ef3-4b74-a509-af9e70908fd3 tempest-TestMinimumBasicScenario-1447275400 tempest-TestMinimumBasicScenario-1447275400-project] HTTP exception thrown: Instance 401ec9c3-4ff3-468a-b334-606c558b8d7a is not attached to volume f3c255b5-9631-420b-ad9d-b251e58e39ae Aug 27 02:46:03.415097 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-fb5eb7c9-7ef3-4b74-a509-af9e70908fd3 tempest-TestMinimumBasicScenario-1447275400 tempest-TestMinimumBasicScenario-1447275400-project] Request method failure captured: Aug 27 02:46:03.415097 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: request: DELETE /compute/v2.1/servers/401ec9c3-4ff3-468a-b334-606c558b8d7a/os-volume_attachments/f3c255b5-9631-420b-ad9d-b251e58e39ae HTTP/1.1 Aug 27 02:46:03.415097 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: Accept: application/json Aug 27 02:46:03.415097 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: Accept-Encoding: identity Aug 27 02:46:03.415097 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: Connection: close Aug 27 02:46:03.415097 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: Content-Length: 0 Aug 27 02:46:03.415097 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: Content-Type: application/json Aug 27 02:46:03.415097 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: Host: 10.222.0.52 Aug 27 02:46:03.415097 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: Openstack-System-Scope: None Aug 27 02:46:03.415097 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: User-Agent: python-urllib3/1.26.11 Aug 27 02:46:03.415097 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-Auth-Token: gAAAAABjCYSikXeQfvJJuR7kaLbpvLsE8jqc4S91OH62MwfKESZpDhWcEpYkPKaBI5lB--ipF54O8HuoVuYwYgLKHDwlzR6E8wbDuhK7lrLTD3Tytvq4ELf2zyk_mcyU7s5lEhTjD3l_Om-_1Ar4xlf_Kp1spAj2nO9c1CSiL-1SYhF4wR5aV2M Aug 27 02:46:03.415097 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-Domain-Id: None Aug 27 02:46:03.415097 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-Domain-Name: None Aug 27 02:46:03.415097 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-Identity-Status: Confirmed Aug 27 02:46:03.415097 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-Is-Admin-Project: True Aug 27 02:46:03.415097 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-Project-Domain-Id: default Aug 27 02:46:03.415097 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-Project-Domain-Name: Default Aug 27 02:46:03.415097 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-Project-Id: 7dd5f20c4a714ee3b555d4aef8b87730 Aug 27 02:46:03.415097 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-Project-Name: tempest-TestMinimumBasicScenario-1447275400 Aug 27 02:46:03.415097 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-Role: reader,member Aug 27 02:46:03.415097 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-Roles: reader,member Aug 27 02:46:03.416942 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-Service-Catalog: [{"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.52:8080", "publicURL": "http://10.222.0.52:8080/v1/AUTH_7dd5f20c4a714ee3b555d4aef8b87730"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52:9696/networking"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/compute/v2/7dd5f20c4a714ee3b555d4aef8b87730"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/image"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/volume/v3/7dd5f20c4a714ee3b555d4aef8b87730"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/identity"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/volume/v3/7dd5f20c4a714ee3b555d4aef8b87730"}]}] Aug 27 02:46:03.416942 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-Tenant: tempest-TestMinimumBasicScenario-1447275400 Aug 27 02:46:03.416942 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-Tenant-Id: 7dd5f20c4a714ee3b555d4aef8b87730 Aug 27 02:46:03.416942 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-Tenant-Name: tempest-TestMinimumBasicScenario-1447275400 Aug 27 02:46:03.416942 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-User: tempest-TestMinimumBasicScenario-1447275400-project Aug 27 02:46:03.416942 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-User-Domain-Id: default Aug 27 02:46:03.416942 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-User-Domain-Name: Default Aug 27 02:46:03.416942 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-User-Id: ba73716667184588b522bc7856404be8 Aug 27 02:46:03.416942 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-User-Name: tempest-TestMinimumBasicScenario-1447275400-project Aug 27 02:46:03.416942 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: method: > Aug 27 02:46:03.416942 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: exception: Instance 401ec9c3-4ff3-468a-b334-606c558b8d7a is not attached to volume f3c255b5-9631-420b-ad9d-b251e58e39ae Aug 27 02:46:03.418288 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: action_args: {'server_id': '401ec9c3-4ff3-468a-b334-606c558b8d7a', 'id': 'f3c255b5-9631-420b-ad9d-b251e58e39ae'} Aug 27 02:46:03.418288 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Aug 27 02:46:03.418288 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 27 02:46:03.418288 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 552, in delete Aug 27 02:46:03.418288 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi bdm = objects.BlockDeviceMapping.get_by_volume_and_instance( Aug 27 02:46:03.418288 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_versionedobjects/base.py", line 184, in wrapper Aug 27 02:46:03.418288 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi result = fn(cls, context, *args, **kwargs) Aug 27 02:46:03.418288 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/block_device.py", line 293, in get_by_volume_and_instance Aug 27 02:46:03.418288 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi raise exception.VolumeBDMNotFound(volume_id=volume_id) Aug 27 02:46:03.418288 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi nova.exception.VolumeBDMNotFound: No volume Block Device Mapping with id f3c255b5-9631-420b-ad9d-b251e58e39ae. Aug 27 02:46:03.418288 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi Aug 27 02:46:03.418288 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 27 02:46:03.418288 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi Aug 27 02:46:03.418288 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 27 02:46:03.418288 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 27 02:46:03.418288 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 27 02:46:03.418288 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Aug 27 02:46:03.418288 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Aug 27 02:46:03.418288 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Aug 27 02:46:03.419888 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Aug 27 02:46:03.419888 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 558, in delete Aug 27 02:46:03.419888 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=msg) Aug 27 02:46:03.419888 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 401ec9c3-4ff3-468a-b334-606c558b8d7a is not attached to volume f3c255b5-9631-420b-ad9d-b251e58e39ae Aug 27 02:46:03.419888 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi Aug 27 02:46:03.419888 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 27 02:46:03.419888 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi Aug 27 02:46:03.419888 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 27 02:46:03.419888 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 27 02:46:03.419888 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 27 02:46:03.419888 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Aug 27 02:46:03.419888 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Aug 27 02:46:03.419888 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 401ec9c3-4ff3-468a-b334-606c558b8d7a is not attached to volume f3c255b5-9631-420b-ad9d-b251e58e39ae Aug 27 02:46:03.419888 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi Aug 27 02:46:03.419888 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-fb5eb7c9-7ef3-4b74-a509-af9e70908fd3 tempest-TestMinimumBasicScenario-1447275400 tempest-TestMinimumBasicScenario-1447275400-project] Returning 404 to user: Instance 401ec9c3-4ff3-468a-b334-606c558b8d7a is not attached to volume f3c255b5-9631-420b-ad9d-b251e58e39ae {{(pid=120863) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Aug 27 02:46:03.421423 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-fb5eb7c9-7ef3-4b74-a509-af9e70908fd3 tempest-TestMinimumBasicScenario-1447275400 tempest-TestMinimumBasicScenario-1447275400-project] 10.222.0.52 "DELETE /compute/v2.1/servers/401ec9c3-4ff3-468a-b334-606c558b8d7a/os-volume_attachments/f3c255b5-9631-420b-ad9d-b251e58e39ae" status: 404 len: 154 microversion: 2.1 time: 0.095704 Aug 27 02:46:03.421423 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 634/1275] 10.222.0.52 () {58 vars in 1416 bytes} [Sat Aug 27 02:46:03 2022] DELETE /compute/v2.1/servers/401ec9c3-4ff3-468a-b334-606c558b8d7a/os-volume_attachments/f3c255b5-9631-420b-ad9d-b251e58e39ae => generated 154 bytes in 97 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Aug 27 02:46:04.564118 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-d3641010-970d-44b5-b33c-1d67f5cea2dd tempest-TestMinimumBasicScenario-1447275400 tempest-TestMinimumBasicScenario-1447275400-project] Calling method '>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:46:04.564878 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-d3641010-970d-44b5-b33c-1d67f5cea2dd tempest-TestMinimumBasicScenario-1447275400 tempest-TestMinimumBasicScenario-1447275400-project] [instance: 401ec9c3-4ff3-468a-b334-606c558b8d7a] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:46:04.573849 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-d3641010-970d-44b5-b33c-1d67f5cea2dd tempest-TestMinimumBasicScenario-1447275400 tempest-TestMinimumBasicScenario-1447275400-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:46:04.574578 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-d3641010-970d-44b5-b33c-1d67f5cea2dd tempest-TestMinimumBasicScenario-1447275400 tempest-TestMinimumBasicScenario-1447275400-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:46:04.575007 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-d3641010-970d-44b5-b33c-1d67f5cea2dd tempest-TestMinimumBasicScenario-1447275400 tempest-TestMinimumBasicScenario-1447275400-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:46:04.575696 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-d3641010-970d-44b5-b33c-1d67f5cea2dd tempest-TestMinimumBasicScenario-1447275400 tempest-TestMinimumBasicScenario-1447275400-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:46:04.576140 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-d3641010-970d-44b5-b33c-1d67f5cea2dd tempest-TestMinimumBasicScenario-1447275400 tempest-TestMinimumBasicScenario-1447275400-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:46:04.576564 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-d3641010-970d-44b5-b33c-1d67f5cea2dd tempest-TestMinimumBasicScenario-1447275400 tempest-TestMinimumBasicScenario-1447275400-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:46:04.602592 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-d3641010-970d-44b5-b33c-1d67f5cea2dd tempest-TestMinimumBasicScenario-1447275400 tempest-TestMinimumBasicScenario-1447275400-project] [instance: 401ec9c3-4ff3-468a-b334-606c558b8d7a] Going to try to terminate instance {{(pid=120864) delete /opt/stack/nova/nova/compute/api.py:2719}} Aug 27 02:46:04.694208 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-d3641010-970d-44b5-b33c-1d67f5cea2dd tempest-TestMinimumBasicScenario-1447275400 tempest-TestMinimumBasicScenario-1447275400-project] 10.222.0.52 "DELETE /compute/v2.1/servers/401ec9c3-4ff3-468a-b334-606c558b8d7a" status: 204 len: 0 microversion: 2.1 time: 0.134138 Aug 27 02:46:04.694880 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 642/1276] 10.222.0.52 () {58 vars in 1239 bytes} [Sat Aug 27 02:46:04 2022] DELETE /compute/v2.1/servers/401ec9c3-4ff3-468a-b334-606c558b8d7a => generated 0 bytes in 136 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Aug 27 02:46:04.704472 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-28de1e56-1ccf-4a5e-a657-2ac241b8965f tempest-TestMinimumBasicScenario-1447275400 tempest-TestMinimumBasicScenario-1447275400-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:46:04.705064 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-28de1e56-1ccf-4a5e-a657-2ac241b8965f tempest-TestMinimumBasicScenario-1447275400 tempest-TestMinimumBasicScenario-1447275400-project] [instance: 401ec9c3-4ff3-468a-b334-606c558b8d7a] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:46:04.712001 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-28de1e56-1ccf-4a5e-a657-2ac241b8965f tempest-TestMinimumBasicScenario-1447275400 tempest-TestMinimumBasicScenario-1447275400-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:46:04.712494 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-28de1e56-1ccf-4a5e-a657-2ac241b8965f tempest-TestMinimumBasicScenario-1447275400 tempest-TestMinimumBasicScenario-1447275400-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:46:04.713314 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-28de1e56-1ccf-4a5e-a657-2ac241b8965f tempest-TestMinimumBasicScenario-1447275400 tempest-TestMinimumBasicScenario-1447275400-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:46:04.714021 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-28de1e56-1ccf-4a5e-a657-2ac241b8965f tempest-TestMinimumBasicScenario-1447275400 tempest-TestMinimumBasicScenario-1447275400-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:46:04.714481 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-28de1e56-1ccf-4a5e-a657-2ac241b8965f tempest-TestMinimumBasicScenario-1447275400 tempest-TestMinimumBasicScenario-1447275400-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:46:04.715101 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-28de1e56-1ccf-4a5e-a657-2ac241b8965f tempest-TestMinimumBasicScenario-1447275400 tempest-TestMinimumBasicScenario-1447275400-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:46:04.853681 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-28de1e56-1ccf-4a5e-a657-2ac241b8965f tempest-TestMinimumBasicScenario-1447275400 tempest-TestMinimumBasicScenario-1447275400-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ba73716667184588b522bc7856404be8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7dd5f20c4a714ee3b555d4aef8b87730', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:46:04.862487 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-28de1e56-1ccf-4a5e-a657-2ac241b8965f tempest-TestMinimumBasicScenario-1447275400 tempest-TestMinimumBasicScenario-1447275400-project] 10.222.0.52 "GET /compute/v2.1/servers/401ec9c3-4ff3-468a-b334-606c558b8d7a" status: 200 len: 1553 microversion: 2.1 time: 0.162299 Aug 27 02:46:04.863132 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 635/1277] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:46:04 2022] GET /compute/v2.1/servers/401ec9c3-4ff3-468a-b334-606c558b8d7a => generated 1553 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:46:05.884890 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-a2621964-e3a5-45b2-8b65-b4342f6221ce tempest-TestMinimumBasicScenario-1447275400 tempest-TestMinimumBasicScenario-1447275400-project] Calling method '>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:46:05.885602 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.compute.api [None req-a2621964-e3a5-45b2-8b65-b4342f6221ce tempest-TestMinimumBasicScenario-1447275400 tempest-TestMinimumBasicScenario-1447275400-project] [instance: 401ec9c3-4ff3-468a-b334-606c558b8d7a] Fetching instance by UUID {{(pid=120864) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:46:05.895410 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-a2621964-e3a5-45b2-8b65-b4342f6221ce tempest-TestMinimumBasicScenario-1447275400 tempest-TestMinimumBasicScenario-1447275400-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:46:05.895900 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-a2621964-e3a5-45b2-8b65-b4342f6221ce tempest-TestMinimumBasicScenario-1447275400 tempest-TestMinimumBasicScenario-1447275400-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:46:05.896326 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-a2621964-e3a5-45b2-8b65-b4342f6221ce tempest-TestMinimumBasicScenario-1447275400 tempest-TestMinimumBasicScenario-1447275400-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:46:05.897027 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-a2621964-e3a5-45b2-8b65-b4342f6221ce tempest-TestMinimumBasicScenario-1447275400 tempest-TestMinimumBasicScenario-1447275400-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:46:05.897508 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-a2621964-e3a5-45b2-8b65-b4342f6221ce tempest-TestMinimumBasicScenario-1447275400 tempest-TestMinimumBasicScenario-1447275400-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:46:05.897925 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [None req-a2621964-e3a5-45b2-8b65-b4342f6221ce tempest-TestMinimumBasicScenario-1447275400 tempest-TestMinimumBasicScenario-1447275400-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:46:06.049560 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.policy [None req-a2621964-e3a5-45b2-8b65-b4342f6221ce tempest-TestMinimumBasicScenario-1447275400 tempest-TestMinimumBasicScenario-1447275400-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ba73716667184588b522bc7856404be8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7dd5f20c4a714ee3b555d4aef8b87730', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120864) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:46:06.059698 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-a2621964-e3a5-45b2-8b65-b4342f6221ce tempest-TestMinimumBasicScenario-1447275400 tempest-TestMinimumBasicScenario-1447275400-project] 10.222.0.52 "GET /compute/v2.1/servers/401ec9c3-4ff3-468a-b334-606c558b8d7a" status: 200 len: 1553 microversion: 2.1 time: 0.179097 Aug 27 02:46:06.060263 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 643/1278] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:46:05 2022] GET /compute/v2.1/servers/401ec9c3-4ff3-468a-b334-606c558b8d7a => generated 1553 bytes in 181 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:46:07.083849 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-6d99d9eb-11b6-4768-b3f5-a277b000c84c tempest-TestMinimumBasicScenario-1447275400 tempest-TestMinimumBasicScenario-1447275400-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:46:07.084819 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-6d99d9eb-11b6-4768-b3f5-a277b000c84c tempest-TestMinimumBasicScenario-1447275400 tempest-TestMinimumBasicScenario-1447275400-project] [instance: 401ec9c3-4ff3-468a-b334-606c558b8d7a] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:46:07.093617 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-6d99d9eb-11b6-4768-b3f5-a277b000c84c tempest-TestMinimumBasicScenario-1447275400 tempest-TestMinimumBasicScenario-1447275400-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:46:07.094288 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-6d99d9eb-11b6-4768-b3f5-a277b000c84c tempest-TestMinimumBasicScenario-1447275400 tempest-TestMinimumBasicScenario-1447275400-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:46:07.094872 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-6d99d9eb-11b6-4768-b3f5-a277b000c84c tempest-TestMinimumBasicScenario-1447275400 tempest-TestMinimumBasicScenario-1447275400-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:46:07.095534 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-6d99d9eb-11b6-4768-b3f5-a277b000c84c tempest-TestMinimumBasicScenario-1447275400 tempest-TestMinimumBasicScenario-1447275400-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:46:07.096057 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-6d99d9eb-11b6-4768-b3f5-a277b000c84c tempest-TestMinimumBasicScenario-1447275400 tempest-TestMinimumBasicScenario-1447275400-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:46:07.096510 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-6d99d9eb-11b6-4768-b3f5-a277b000c84c tempest-TestMinimumBasicScenario-1447275400 tempest-TestMinimumBasicScenario-1447275400-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:46:07.189821 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.policy [None req-6d99d9eb-11b6-4768-b3f5-a277b000c84c tempest-TestMinimumBasicScenario-1447275400 tempest-TestMinimumBasicScenario-1447275400-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ba73716667184588b522bc7856404be8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7dd5f20c4a714ee3b555d4aef8b87730', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120863) authorize /opt/stack/nova/nova/policy.py:199}} Aug 27 02:46:07.200831 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-6d99d9eb-11b6-4768-b3f5-a277b000c84c tempest-TestMinimumBasicScenario-1447275400 tempest-TestMinimumBasicScenario-1447275400-project] 10.222.0.52 "GET /compute/v2.1/servers/401ec9c3-4ff3-468a-b334-606c558b8d7a" status: 200 len: 1511 microversion: 2.1 time: 0.122542 Aug 27 02:46:07.201482 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 636/1279] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:46:07 2022] GET /compute/v2.1/servers/401ec9c3-4ff3-468a-b334-606c558b8d7a => generated 1511 bytes in 124 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 27 02:46:07.220230 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [req-fb19c02c-8299-4c24-83ad-a2da2af30ed8 req-bc1896c6-964c-4e60-a1d7-42de8c7615f1 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "401ec9c3-4ff3-468a-b334-606c558b8d7a", "name": "network-vif-deleted", "tag": "974dc3f6-d8bf-4ab5-a284-a4ecb61b8866"}]} {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 27 02:46:07.230577 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [req-fb19c02c-8299-4c24-83ad-a2da2af30ed8 req-bc1896c6-964c-4e60-a1d7-42de8c7615f1 service nova] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:46:07.231060 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [req-fb19c02c-8299-4c24-83ad-a2da2af30ed8 req-bc1896c6-964c-4e60-a1d7-42de8c7615f1 service nova] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:46:07.231506 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG oslo_concurrency.lockutils [req-fb19c02c-8299-4c24-83ad-a2da2af30ed8 req-bc1896c6-964c-4e60-a1d7-42de8c7615f1 service nova] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:46:07.243736 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.compute.server_external_events [req-fb19c02c-8299-4c24-83ad-a2da2af30ed8 req-bc1896c6-964c-4e60-a1d7-42de8c7615f1 service nova] Creating event network-vif-deleted:974dc3f6-d8bf-4ab5-a284-a4ecb61b8866 for instance 401ec9c3-4ff3-468a-b334-606c558b8d7a on hv-cinder-84630 Aug 27 02:46:07.249881 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [req-fb19c02c-8299-4c24-83ad-a2da2af30ed8 req-bc1896c6-964c-4e60-a1d7-42de8c7615f1 service nova] 10.222.0.52 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.034114 Aug 27 02:46:07.250454 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 644/1280] 10.222.0.52 () {64 vars in 1317 bytes} [Sat Aug 27 02:46:07 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 36 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 27 02:46:08.224104 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-cdafde63-2c72-4899-bc39-3c5fe093ba74 tempest-TestMinimumBasicScenario-1447275400 tempest-TestMinimumBasicScenario-1447275400-project] Calling method '>' {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:46:08.225015 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.compute.api [None req-cdafde63-2c72-4899-bc39-3c5fe093ba74 tempest-TestMinimumBasicScenario-1447275400 tempest-TestMinimumBasicScenario-1447275400-project] [instance: 401ec9c3-4ff3-468a-b334-606c558b8d7a] Fetching instance by UUID {{(pid=120863) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 27 02:46:08.232040 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-cdafde63-2c72-4899-bc39-3c5fe093ba74 tempest-TestMinimumBasicScenario-1447275400 tempest-TestMinimumBasicScenario-1447275400-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:46:08.232529 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-cdafde63-2c72-4899-bc39-3c5fe093ba74 tempest-TestMinimumBasicScenario-1447275400 tempest-TestMinimumBasicScenario-1447275400-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:46:08.232967 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-cdafde63-2c72-4899-bc39-3c5fe093ba74 tempest-TestMinimumBasicScenario-1447275400 tempest-TestMinimumBasicScenario-1447275400-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:46:08.233701 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-cdafde63-2c72-4899-bc39-3c5fe093ba74 tempest-TestMinimumBasicScenario-1447275400 tempest-TestMinimumBasicScenario-1447275400-project] Acquiring lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 27 02:46:08.234153 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-cdafde63-2c72-4899-bc39-3c5fe093ba74 tempest-TestMinimumBasicScenario-1447275400 tempest-TestMinimumBasicScenario-1447275400-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 27 02:46:08.234576 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG oslo_concurrency.lockutils [None req-cdafde63-2c72-4899-bc39-3c5fe093ba74 tempest-TestMinimumBasicScenario-1447275400 tempest-TestMinimumBasicScenario-1447275400-project] Lock "41dc7c6b-010c-43dc-bd7e-0984fcde0c35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120863) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 27 02:46:08.249293 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.wsgi [None req-cdafde63-2c72-4899-bc39-3c5fe093ba74 tempest-TestMinimumBasicScenario-1447275400 tempest-TestMinimumBasicScenario-1447275400-project] HTTP exception thrown: Instance 401ec9c3-4ff3-468a-b334-606c558b8d7a could not be found. Aug 27 02:46:08.250845 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-cdafde63-2c72-4899-bc39-3c5fe093ba74 tempest-TestMinimumBasicScenario-1447275400 tempest-TestMinimumBasicScenario-1447275400-project] Request method failure captured: Aug 27 02:46:08.250845 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: request: GET /compute/v2.1/servers/401ec9c3-4ff3-468a-b334-606c558b8d7a HTTP/1.1 Aug 27 02:46:08.250845 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: Accept: application/json Aug 27 02:46:08.250845 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: Accept-Encoding: identity Aug 27 02:46:08.250845 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: Connection: close Aug 27 02:46:08.250845 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: Content-Length: 0 Aug 27 02:46:08.250845 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: Content-Type: application/json Aug 27 02:46:08.250845 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: Host: 10.222.0.52 Aug 27 02:46:08.250845 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: Openstack-System-Scope: None Aug 27 02:46:08.250845 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: User-Agent: python-urllib3/1.26.11 Aug 27 02:46:08.250845 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-Auth-Token: gAAAAABjCYSikXeQfvJJuR7kaLbpvLsE8jqc4S91OH62MwfKESZpDhWcEpYkPKaBI5lB--ipF54O8HuoVuYwYgLKHDwlzR6E8wbDuhK7lrLTD3Tytvq4ELf2zyk_mcyU7s5lEhTjD3l_Om-_1Ar4xlf_Kp1spAj2nO9c1CSiL-1SYhF4wR5aV2M Aug 27 02:46:08.250845 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-Domain-Id: None Aug 27 02:46:08.250845 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-Domain-Name: None Aug 27 02:46:08.250845 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-Identity-Status: Confirmed Aug 27 02:46:08.250845 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-Is-Admin-Project: True Aug 27 02:46:08.250845 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-Project-Domain-Id: default Aug 27 02:46:08.250845 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-Project-Domain-Name: Default Aug 27 02:46:08.250845 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-Project-Id: 7dd5f20c4a714ee3b555d4aef8b87730 Aug 27 02:46:08.250845 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-Project-Name: tempest-TestMinimumBasicScenario-1447275400 Aug 27 02:46:08.250845 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-Role: reader,member Aug 27 02:46:08.250845 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-Roles: reader,member Aug 27 02:46:08.253054 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-Service-Catalog: [{"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.52:8080", "publicURL": "http://10.222.0.52:8080/v1/AUTH_7dd5f20c4a714ee3b555d4aef8b87730"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52:9696/networking"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/compute/v2/7dd5f20c4a714ee3b555d4aef8b87730"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/image"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/volume/v3/7dd5f20c4a714ee3b555d4aef8b87730"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/identity"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/volume/v3/7dd5f20c4a714ee3b555d4aef8b87730"}]}] Aug 27 02:46:08.253054 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-Tenant: tempest-TestMinimumBasicScenario-1447275400 Aug 27 02:46:08.253054 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-Tenant-Id: 7dd5f20c4a714ee3b555d4aef8b87730 Aug 27 02:46:08.253054 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-Tenant-Name: tempest-TestMinimumBasicScenario-1447275400 Aug 27 02:46:08.253054 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-User: tempest-TestMinimumBasicScenario-1447275400-project Aug 27 02:46:08.253054 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-User-Domain-Id: default Aug 27 02:46:08.253054 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-User-Domain-Name: Default Aug 27 02:46:08.253054 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-User-Id: ba73716667184588b522bc7856404be8 Aug 27 02:46:08.253054 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: X-User-Name: tempest-TestMinimumBasicScenario-1447275400-project Aug 27 02:46:08.253054 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: method: > Aug 27 02:46:08.253054 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: exception: Instance 401ec9c3-4ff3-468a-b334-606c558b8d7a could not be found. Aug 27 02:46:08.253054 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: action_args: {'id': '401ec9c3-4ff3-468a-b334-606c558b8d7a'} Aug 27 02:46:08.254191 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: {{(pid=120863) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Aug 27 02:46:08.254191 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 27 02:46:08.254191 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Aug 27 02:46:08.254191 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Aug 27 02:46:08.254191 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Aug 27 02:46:08.254191 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Aug 27 02:46:08.254191 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Aug 27 02:46:08.254191 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Aug 27 02:46:08.254191 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Aug 27 02:46:08.254191 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Aug 27 02:46:08.254191 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 401ec9c3-4ff3-468a-b334-606c558b8d7a could not be found. Aug 27 02:46:08.254191 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi Aug 27 02:46:08.254191 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 27 02:46:08.254191 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi Aug 27 02:46:08.254191 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 27 02:46:08.254191 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 27 02:46:08.254191 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 27 02:46:08.254191 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Aug 27 02:46:08.254191 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Aug 27 02:46:08.254191 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Aug 27 02:46:08.255738 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Aug 27 02:46:08.255738 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Aug 27 02:46:08.255738 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi instance = self._get_server( Aug 27 02:46:08.255738 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Aug 27 02:46:08.255738 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Aug 27 02:46:08.255738 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Aug 27 02:46:08.255738 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Aug 27 02:46:08.255738 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 401ec9c3-4ff3-468a-b334-606c558b8d7a could not be found. Aug 27 02:46:08.255738 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi Aug 27 02:46:08.255738 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 27 02:46:08.255738 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi Aug 27 02:46:08.255738 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 27 02:46:08.255738 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 27 02:46:08.255738 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 27 02:46:08.255738 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Aug 27 02:46:08.255738 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Aug 27 02:46:08.255738 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 401ec9c3-4ff3-468a-b334-606c558b8d7a could not be found. Aug 27 02:46:08.255738 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: ERROR nova.api.openstack.wsgi Aug 27 02:46:08.257346 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: DEBUG nova.api.openstack.wsgi [None req-cdafde63-2c72-4899-bc39-3c5fe093ba74 tempest-TestMinimumBasicScenario-1447275400 tempest-TestMinimumBasicScenario-1447275400-project] Returning 404 to user: Instance 401ec9c3-4ff3-468a-b334-606c558b8d7a could not be found. {{(pid=120863) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Aug 27 02:46:08.257346 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: INFO nova.api.openstack.requestlog [None req-cdafde63-2c72-4899-bc39-3c5fe093ba74 tempest-TestMinimumBasicScenario-1447275400 tempest-TestMinimumBasicScenario-1447275400-project] 10.222.0.52 "GET /compute/v2.1/servers/401ec9c3-4ff3-468a-b334-606c558b8d7a" status: 404 len: 111 microversion: 2.1 time: 0.032049 Aug 27 02:46:08.257346 dv-cinder-846303-3-iscsi devstack@n-api.service[120863]: [pid: 120863|app: 0|req: 637/1281] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Aug 27 02:46:08 2022] GET /compute/v2.1/servers/401ec9c3-4ff3-468a-b334-606c558b8d7a => generated 111 bytes in 33 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Aug 27 02:46:08.262508 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: DEBUG nova.api.openstack.wsgi [None req-30718d9d-f6dc-48f1-a310-d922b9c5e4eb tempest-TestMinimumBasicScenario-1447275400 tempest-TestMinimumBasicScenario-1447275400-project] Calling method '.version_select at 0x7f6b1ea10310>' {{(pid=120864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 27 02:46:08.285499 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: INFO nova.api.openstack.requestlog [None req-30718d9d-f6dc-48f1-a310-d922b9c5e4eb tempest-TestMinimumBasicScenario-1447275400 tempest-TestMinimumBasicScenario-1447275400-project] 10.222.0.52 "DELETE /compute/v2.1/os-keypairs/tempest-TestMinimumBasicScenario-970568034" status: 202 len: 0 microversion: 2.1 time: 0.027756 Aug 27 02:46:08.286097 dv-cinder-846303-3-iscsi devstack@n-api.service[120864]: [pid: 120864|app: 0|req: 645/1282] 10.222.0.52 () {58 vars in 1269 bytes} [Sat Aug 27 02:46:08 2022] DELETE /compute/v2.1/os-keypairs/tempest-TestMinimumBasicScenario-970568034 => generated 0 bytes in 29 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0)